Assessing acceptance in mindfulness with direct-worded items: The development and initial validation of the Athlete Mindfulness Questionnaire.

PURPOSE: The purpose of the current study was to develop and validate a sport-specific mindfulness measure, the Athlete Mindfulness Questionnaire (AMQ), through 5 related studies using 4 separate samples of Chinese athletes. The AMQ is a 3-factor measure designed to assess mindfulness that reflects present-moment attention, awareness, and acceptance in a sport context.
METHODS: In Study 1, an initial pool of items was generated based on previous literature, existing mindfulness scales, as well as interviews with and feedback from the athletes, coaches, and mindfulness experts. Initial support for the 3-factor structure of the AMQ was established via exploratory factor analysis in Study 2, and cross-validated through confirmatory factor analysis in Studies 3 and 4. In Study 5, a modified 3-factor AMQ with direct-worded acceptance items was examined in a fourth independent sample.
RESULTS: Convergent and concurrent validities of the acceptance subscale failed to be established in Studies 3 and 4 which may be due to the inattention and confusion of the athletes whilst interpreting the reverse-worded items. A modified 16-item AMQ in Study 5 displayed satisfactory model fit and acceptable internal consistencies. Most importantly, convergent and concurrent validities of the 16-item AMQ were supported. The 3 subscales showed significant positive associations with mindfulness, flow, well-being, and positive affect and significant negative associations with experiential avoidance, burnout, and negative affect.
CONCLUSION: The AMQ is a psychometrically sound measure of mindfulness in a sport context. The importance of using direct-worded acceptance items is discussed.

Introduction

Mindfulness is described as frequent receptive attention to internal and external stimuli as they occur in Buddhist philosophy and practice. It has been adopted by sport psychology researchers and practitioners to better understand and enhance the performance of athletes for over a decade. The characteristic of not being passively resigned to, or dissociated from, but actively engaged in the observed experience of mindfulness, is consistent with the concept of ideal performance in a sport context. In this context, athletes must actively deal with or coexist with both internal (e.g., disruptive thoughts and emotions) and external stimuli (environmental or situational factors), while focusing on present-moment performance.3, 4, 5

In a sport context, the ability to self-regulate attention to present performance, awareness of internal and external experiences of bodily sensations, emotions, thoughts, and cognitions, along with an accepting and non-judgmental attitude towards these experiences have been emphasized and cultivated under the general framework of mindfulness training.6, 7 Accordingly, mindfulness-based approaches to performance enhancement and the general well-being of athletes have been developed.6, 7, 8 Preliminary evidence of the effectiveness of mindfulness interventions for athletes has revealed sport performance enhancement, increased mindfulness states, as well as other positive states such as flow, sport confidence, and well-being; and the decrease of negative affect such as anxiety, burnout, worries, and perfectionism.8, 9, 10, 11, 12

On the other hand, examination of the associations between mindfulness and relevant key variables, such as flow, task-orientation, satisfaction, and sport performance in athletic populations, informs conceptual and intervention considerations in a sport context. Yet, existing evidence has not been built on measurements that had been specifically developed for athletes. Given that the sporting context is quite different from the context of daily life, in terms of the intensity of stimulus and the requirements of attention and awareness, sport-specific mindfulness scales are needed in order to more accurately assess mindfulness among athletic population. As such, Thienot and colleagues developed a 15-item Mindfulness Inventory for Sport (MIS), including 3 factors: awareness, non-judgmental attitude, and refocusing.

In the MIS, athletes are asked to assess their awareness of private experiences and their refocusing ability and non-judgmental attitude after they become aware of these experiences. Yet, it is believed that attention and awareness should be viewed as 2 inter-related components instead of one component occurring after another. In other words, although awareness can bring attention to the present moment when the mind wanders, the nature of mindfulness is the present-moment attention and the awareness of the present-moment experiences even within a sport context. Furthermore, the original conceptualization of mindfulness and mindfulness practice20, 21 stated that the present-moment attention contains refocusing capability. Therefore, the sole assessment of refocusing skills by neglecting the capability of sustaining attention on the present-moment performance might lead to the omission of important information.

To overcome the limitation in Thienot and colleagues' conceptualization of mindfulness, the attentional component in the current research was conceptualized as comprising (a) sustained attention on the present-moment experience and (b) bringing the attention back to the present moment. In addition, the present-moment attention and non-judgment (or acceptance) are viewed as 2 independent components of mindfulness that occur simultaneously, rather than 2 components which occur subsequent to awareness. Grounded in the framework of mindfulness within the context of sport, we define mindfulness in terms of 3 components: present-moment attention, awareness, and acceptance.

In summary, the current research aimed to develop and preliminarily validate a scale that assesses athletes' dispositional mindfulness, entitled the Athlete Mindfulness Questionnaire (AMQ), via a series of 5 studies. The aim of Study 1 was to create a pool of items that captured the dispositional mindfulness of Chinese athletes, to gauge how applicable the items of the scale were in a sport context, and to provide evidence for the scale's content validity. The aim of Study 2 was to analyze the factorial composition of the items generated in Study 1 via an exploratory factor analysis (EFA). The aim of Study 3 was to cross-validate the findings of Study 2 using confirmatory factor analysis (CFA) with another sample of athletes, and further refine the structure of the AMQ if necessary. The aim of Study 4 was to use another sample to cross-validate the structure of the model supported in Study 3. The convergent and concurrent validities of the AMQ were also examined in Studies 3 and 4. The aim of Study 5 was to validate the structure of a modified AMQ with direct-worded acceptance items, and examine its convergent and concurrent validity through testing its relationships with relevant concepts.

Methods

Participants

In Study 1, participants were 27 Chinese athletes (16 males and 11 females) and 8 Chinese coaches (6 males and 2 females); both were drawn from 5 sports (diving, gymnastics, synchronized swimming, table tennis, and wushu). The coaches' coaching experience ranged from 1 to 25 years (10.13 ± 9.28, mean ± SD). The athletes were between 18 and 27 years old (20.93 ± 2.29), competed at national (n = 15) or international (n = 12) levels, and their competitive experience ranged from 7 to 23 years (13.37 ± 4.34). A panel of 7 Chinese mindfulness experts was also invited to review the content validity of the items. Table 1 shows participant details of Studies 2–5.

Table 1

Participant characteristics.

	Study 2	Study 3	Study 4	Study 5
n	271	357	295	379
Male	135	208	158	217
Female	136	148	137	162
Unknown gender	–	1	–	–
Age (year)a	21.55 ± 3.15	21.28 ± 3.94	21.34 ± 3.19	19.59 ± 3.47
Age range (year)	18–33	17–45	17–37	16–35
Number of sports	18	27	20	20
Sport type
Teamb	62	103	102	97
Individualc	209	254	193	282
Competitive level
National	176	238	195	285
Internationald	95	119	100	94
Time competing (year)a	9.03 ± 4.29	6.91 ± 4.13	7.33 ± 3.83	6.33 ± 3.95
Range of time competing (year)	1–22	1–27	1–23	1–20
Number of data collection venuese	2	6	4	4

mean ± SD.

Team sports including basketball, handball, soccer, synchronized swimming, volleyball, water polo, and so on.

Individual sports including archery, athletics, badminton, boxing, diving, fencing, gymnastics, judo, shooting, swimming, taekwondo, weight lifting, wrestling, wushu, and so on.

Currently compete or have competed at an international level.

Regional training centers in the Mainland of China or Chinese national championships competition venues.

Measures

In Studies 2–5, AMQ items were rated on a 5-point Likert scale ranging from 1 (never true) to 5 (always true).

In Study 3, 4 instruments were used to build the convergent and concurrent validities of AMQ: (1) The Mindful Attention Awareness Scale (MAAS) is a unidimensional scale with 15 items that are rated on a 6-point Likert scale from 1 (almost always) to 6 (almost never); higher scores indicate higher levels of mindfulness. The Chinese translated MAAS demonstrated adequate reliability and validity among an athletic sample. (2) The Acceptance and Action Questionnaire II (AAQ-II) is a 7-item self-report measure with items that are rated on a 7-point Likert scale, from 1 (never true) to 7 (always true); higher scores indicate higher levels of experiential avoidance. The AAQ-II demonstrated adequate reliability and validity in Chinese athletes. (3) The Short Dispositional Flow Scale (SDFS) is a 9-item scale, with 1 item measures each of the 9 flow dimensions. The SDFS is rated on a 5-point Likert scale, ranging from 1 (never) to 5 (always). The SDFS demonstrated adequate reliability and validity among a Chinese athletic sample. (4) The Training and Competition Well-being Scale (TCWS) is a 6-item scale used to assess Chinese athletes' subjective well-being in their training and competition. All items on the TCWS are scored on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree).

In Study 4, 5 instruments, subsequent to the procedure of translation and back-translation, were used to test the concurrent validities of the AMQ: (1) The Athlete Burnout Questionnaire (ABQ) is a 15-item self-report instrument measuring 3 burnout subscales: emotional/physical exhaustion, reduced sense of accomplishment, and sport devaluation. Respondents were asked to indicate how often they felt or thought a certain way during the current season using a 5-point Likert scale ranging from 1 (almost never) to 5 (almost always). (2) The Subjective Vitality Scale (SVS) is a 6-item unidimensional instrument used to measure individuals' levels of subjective vitality. Responses were provided on a 7-point Likert scale ranging from 1 (not at all true) to 7 (very true). (3) The International Positive and Negative Affect Schedule (PANAS) Short Form (I-PANAS-SF) is a short form of PANAS used to measure individuals' positive and negative affect. It included 10 items, 5 items each for positive affect (PA) and negative affect (NA) subscales. Respondents were requested to rate the statement on a 5-point Likert scale ranging from 1 (never) to 5 (always) by comparing themselves during the past week with their “usual selves”. (4) The Sport Enjoyment Scale (SES) is a 4-item unidimensional instrument used in this study to measure athletes' positive affective response to their sport experience that reflects generalized feelings such as pleasure, liking, and fun. Responses were provided on a 5-point Likert scale ranging from 1 (not at all) to 5 (very much). (5) The Sport Competition Anxiety Test (SCAT) is a 15-item self-report instrument measuring one's tendency to perceive competitive situations as threatening, which can lead to increased intensity of one's state-based reaction to competitive situations. The SCAT utilized a 3-point Likert scale (hardly ever, sometimes, and often). It demonstrated adequate reliability and validity among a Chinese athletic sample.

In Study 5, 6 measures were tested, including the MAAS, the AAQ-II, the SDFS, and the TCWS as outlined in Study 3, as well as the ABQ and the I-PANAS-SF as described in Study 4.

Procedure

Study 1

Ethical approval was obtained from the Ethics Committee of the Hong Kong Baptist University for each of the following studies. At the first stage, items relating to mindfulness in sport context were developed. Feasible items were selected and modified from some of the well-established self-report instruments to form the candidate items of the initial item pool. To maximize the clarity, specificity, and brevity of items, guidelines for item wording suggested by DeVellis were closely followed.

At the second stage, coaches and athletes were recruited via the sport psychology consultants who were working with them at the time of data collection. Five semi-structured interviews and 1 focus group (3 wushu coaches) were conducted with the coaches. The duration of the coach interviews lasted approximately 30–60 min. Five focus groups were conducted with the athletes for approximately 90–110 min. The purpose of these interviews was to identify and generate mindfulness characteristics occurring in the sport environment.

At the third stage, a list of items from the first 2 stages were presented to athletes using a dichotomous scale (applicable vs. inapplicable), in which they were instructed to assess the relevance of each item to the sport context. Items deemed inapplicable by 33% or more of the athletes were eliminated. The athletes were also asked to rate the clarity of the applicable items using a 7-point Likert scale (1 = not at all clear to 7 = extremely clear). Items rated below 5 were taken as problematic, and the athletes were encouraged to suggest alternative wordings for these problematic items.

At the final stage, a reduced pool of items, which had been established following the quantitative review, were sent out to 7 experts nationwide via email to invite their expert opinions. Two steps were taken in this stage. In the first step, the experts were asked to rate how representative each item was in its ability to reflect the related components of mindfulness, using a 4-point Likert scale from 1 (not relevant) to 4 (highly relevant). Experts were also asked to suggest alternative or additional items. Subsequently, after necessary modifications, 4 of the 7 experts were asked to rate the representation of each item using the same scale as in the second step.

Studies 2–5

Coaches and team managers were contacted directly to seek the permission to approach their athletes and at this point the purpose of the study was explained. Upon receiving oral approval, the primary researcher approached the athletes to arrange distribution of the questionnaires and the data collection. Informed consent was received from athletes and standardized instructions were given to them. Athletes were clearly informed of their voluntary and anonymous participation in the study.

Data analysis

Content validity

In Study 1, the item-level content validity index (I-CVI)35, 36 was calculated for each item by dividing the number of experts who rated the item as quite relevant or highly relevant (rating 3 and 4 given by experts) by the total number of experts taking part in the rating. When an expert panel consists of 6 or more reviewers, I-CVIs lower than the 0.78 criteria are believed to be less satisfactory and can be removed. To calculate the scale-level content validity index, all the I-CVIs were averaged.

EFA

In Study 2, data were analyzed using an EFA in Mplus 7, to identify the underlying dimensions of the AMQ. Owing to the documented shortcomings associated with maximum likelihood (ML) for estimating factor analysis models for ordinal data, a polychoric correlation matrix using weighted least squares mean- and variance-adjusted (WLSMV) estimation procedure with an oblique Geomin rotation was carried out. The percentages of missing data for the AMQ (0.165%) were negligible, and all missing data were treated using pairwise deletion to produce unbiased estimates for the parameters and their standard errors. The Geomin rotation was selected given that it was designed to minimize cross-loadings while producing statistically significant factor loadings on the primary factors, which is likely to generate cleaner factor structures comparable to confirmatory factor analysis (CFA).

Following the recommendation of Schmitt, the number of items was determined with parallel analysis in Mplus 7, and then evaluated using the model fit indices. Although multiple fit indices such as the comparative fit index (CFI), the Tucker–Lewis index (TLI), the standardized root mean square residual (SRMR), and the root mean square error of approximation (RMSEA) were only validated for CFA models and had not been formally determined for EFA, they were used to provide additional support for the initial EFA model we obtained. In general, a cut-off value of CFI and TLI greater than 0.90 is considered as adequate model fit, and greater than 0.95 and above is suggested to indicate an excellent fit. For the SRMR and RMSEA along with its associated 90% confidence interval (CI), values closed to or less than 0.08 and 0.06 are indicative of adequate model fit, respectively.

In terms of interpreting the extracted factors, items were removed in the following order: (a) items with large factor pattern loadings on the incorrect factor (i.e., ≥0.40); (b) items with high cross-loadings, namely, secondary loadings > 0.30; and (c) items with primary factor loadings ≤ 0.40, indicating that items did not load on any factor. Items were removed independently based on the item severity following a sequence of factor analyses until an approximate simple structure was obtained. Bivariate correlations between the subscales were also examined and small-to-moderate relationship values would indicate that the subscales are measuring related but distinct constructs.

CFA

In Studies 3–5, the AMQ was analyzed via CFA in Mplus 7 using the polychoric correlation matrix and the WLSMV estimator, in order to cross-validate the findings of the EFA. The adequacy of the CFA model to the data was evaluated based on model fit statistics (multiple fit indices) and estimated standardized factor-loading magnitudes. The fit statistics (χ2, CFI, TLI, SRMR, and RMSEA) outlined in Study 2 for the EFA and the weighted root mean square residual (WRMR) were employed to evaluate model fit for the CFA. A WRMR value of close to or less than 1.0 has been suggested as indicative of adequate model fit. In addition, items with factor loadings below 0.40 and large absolute values of standardized residuals (>2.00) were considered for removal.

The percentages of missing data were negligible in Study 3 (0.193%), Study 4 (0.282%), and Study 5 (0.543%) and missing data were treated use pairwise deletion. The low cell count of “not at all” response option was recoded from a “1” to a “2” to create a more symmetrical distribution, increase model stability, improve threshold estimation, and eliminate convergence problems.

Descriptive statistics and internal consistency reliabilities using composite reliability were calculated. Discriminant validity was examined through the inspection of factor correlations, as well as the existence of an alternative single-factor model. Concurrent and convergent validities were examined through SPSS statistics Version 21.0 (IBM Corp., Armonk, NY, USA).

Results

Study 1: item generation and content validity

Initially, 64 items were generated by adapting the items of existing instruments into the context of athletes' training and competition. A total of 23 new items were added to reflect additional mindfulness descriptions that had been suggested by coaches and athletes, and subsequently, an initial pool of 87 items was obtained. The relevance and clarity of each item was also evaluated by the athletes, and as a result, 41 items were deemed inapplicable to the context of sport and were eliminated, whereas 18 items were slightly modified to improve their clarity and broaden their applicability across sports. Based on the ratings of the resultant pool of 46 items by 7 experts, 10 items that displayed an I-CVI of 0.71 (5/7) or below were deleted, and the remaining items which exhibited I-CVIs ranging from 0.86 (6/7) to 1.00 (7/7) were retained. Minor modifications were made to the wording of 7 items and 2 new items were added. At the second-round rating, 4 out of these 7 experts were invited again to rate the relevance of the reduced pool of 38 items and a satisfactory S-CVI/Ave of 0.96 was obtained.

Study 2: examining the factor structure of AMQ

The initial EFA with 38 items revealed a 3-factor solution based on the parallel analysis (mean eigenvalue) using Mplus 7, although 10 eigenvalues were greater than 1. Factors 1, 2, and 3 explained 21.26%, 9.07%, and 4.32% of variance, respectively. Additional support for the 3-factor model was obtained using the model fit statistics, the 3-factor model indicate an adequate fit, where χ2(592) = 874.02, p < 0.001, CFI = 0.94, TLI = 0.93, SRMR = 0.05, RMSEA = 0.04 (90%CI: 0.036–0.048).

By employing the aforementioned criteria to examine the factor loading matrix, 22 items were removed in a sequence of factor analyses. The final EFA solution contained 16 items that loaded on to 3 factors, with Factors 1, 2, and 3 explaining 31.96%, 10.93%, and 7.54% of variance, respectively. Analysis of item content and using constructs previously identified in the literature, Factor 1 was labeled as present-moment attention (6 items), Factor 2 was labeled as awareness (5 items), and Factor 3 was labeled as acceptance (5 items) (Table 2). The inter-factor correlations were moderate to large in magnitude (Table 3). Based on the standardized factor loadings of the final set of items, the composite reliability was calculated (Table 3).

Table 2

Factor loading matrix, factor loadings (λ), error variances (θ), item means (M), and standard deviations (SD) of the AMQ (Studies 2, 3, and 4).

AMQ subscale and item content	Study 2			Study 3				Study 4
	F1	F2	F3	M	SD	λ	θ	M	SD	λ	θ
F1. Present-moment attention
1. I can maintain my attention on my training.	0.68*	−0.12	−0.03	4.01	0.74	0.51	0.74	3.96	0.67	0.60	0.64
4. When I find myself distracted, I gently bring my attention back to my training.	0.49*	0.12	0.00	3.53	0.88	0.73	0.47	3.58	0.92	0.65	0.58
9. When I am about to win the competition, I can still focus on the things I am doing.	0.69*	−0.17	0.03	–	–	–	–	–	–	–	–
20. I can easily sustain my attention on the competition.	0.70*	0.02	0.08	3.82	0.87	0.56	0.68	3.91	0.89	0.54	0.71
28. When I feel muscular pain during training, I can still maintain my attention on the things I should do.	0.56*	0.08	−0.03	3.70	0.91	0.56	0.69	3.64	0.86	0.60	0.64
33. If I notice that my mind is wandering, I can quickly get back to focusing on my training or competition.	0.59*	−0.02	−0.07	3.51	0.92	0.61	0.62	3.56	0.90	0.71	0.45
F2. Awareness
14. When something unexpected happens during training or competition, I am aware of my emotional state.	0.04	0.52*	−0.05	3.54	0.87	0.61	0.62	3.48	0.80	0.50	0.75
15. I am aware of the changes inside my body during competition, for example when my heart beats faster or when my muscles become tense.	−0.02	0.52*	−0.13	–	–	–	–	–	–	–	–
24. When the situation changes during the competition, I am aware of the thoughts and ideas that flashed across my mind.	0.00	0.82*	0.14	3.57	0.91	0.64	0.60	3.43	0.89	0.58	0.67
27. When the competition process is totally beyond my expectations, I am aware of my physical reactions and changes.	0.01	0.60*	0.04	3.51	0.99	0.48	0.77	3.46	0.93	0.52	0.73
38. During training or competition, I can be immediately aware of my emotional changes.	0.15*	0.48*	−0.02	3.67	0.91	0.65	0.58	3.60	0.88	0.59	0.66
F3. Acceptance (reverse-worded item)
13. When certain thoughts and ideas emerge during training or competition, I tell myself that I should not think these thought.	0.02	−0.25*	0.46*	3.37	0.93	0.60	0.64	3.46	0.94	0.66	0.57
16. I try to get rid of annoying thoughts and ideas during training or competition.	−0.08	0.02	0.66*	3.38	0.97	0.58	0.66	3.41	0.93	0.65	0.57
19. During training or competition, I tell myself that I should not have certain thoughts.	−0.02	0.00	0.70*	3.24	0.99	0.82	0.33	3.29	0.94	0.61	0.63
22. When I experience unpleasant emotions during training or competition, I try to control or eliminate them.	−0.30*	−0.05	0.42*	–	–	–	–	–	–	–	–
23. I try to distract or divert my attention to make bad memories during training or competition disappear.	−0.01	−0.21*	0.44*	–	–	–	–	–	–	–	–

Notes: Numbers in bold face indicate primary loadings from Study 2. All factor loadings from Studies 3 and 4 are statistically significant at p < 0.01.

Abbreviations: AMQ = Athlete Mindfulness Questionnaire; F = factor.

p < 0.05.

Table 3

Factor correlations and internal consistency reliabilities of the AMQ (Studies 2, 3, and 4).

	Study 2			Study 3			Study 4
	1	2	3	1	2	3	1	2	3
1. Present-moment attention	0.79	–	–	0.73	–	–	0.76	–	–
2. Awareness	0.49	0.73	–	0.45	0.69	–	0.35	0.63	–
3. Acceptance	−0.48	−0.58	0.67	−0.29	−0.45	0.71	−0.31	−0.53	0.68

Notes: All inter-factor correlations are statistically significant at p < 0.01. Composite reliability coefficients are presented on the diagonal of the factor correlation matrix.

Abbreviation: AMQ = Athlete Mindfulness Questionnaire.

Study 3: validating the factor structure and providing validity evidence of AMQ

The CFA results suggested a marginal good fit to the data, but indicated room for improvement: χ2(101) = 301.43, p < 0.001, CFI = 0.90, TLI = 0.88, WRMR = 1.22, RMSEA = 0.08 (90%CI: 0.065–0.084). Large modification indices suggested that several items had secondary pattern coefficients that should be addressed, and 4 items were, therefore, excluded sequentially. Excluding these 4 items improved the fit of the model to the data: χ2(51) = 123.14, p < 0.001, CFI = 0.95, TLI = 0.93, WRMR = 0.94, RMSEA = 0.06 (90%CI: 0.049–0.077). The model included 3 factors: present-moment attention (5 items), awareness (4 items), and acceptance (3 items). In addition, a 1-factor model was tested, which produced a very poor fit to the data: χ2(54) = 277.83, p < 0.001, CFI = 0.83, TLI = 0.80, WRMR = 1.47, RMSEA = 0.11 (90%CI: 0.095–0.120). Table 2 shows item means, standard deviations, standardized factor loadings and residuals, and Table 3 shows factor correlations and internal consistency reliabilities.

Convergent and concurrent validities

With regard to the convergent validity, present-moment attention and awareness correlated significantly with mindfulness measured by MAAS, but not acceptance. On the concurrent validity, present-moment attention and awareness correlated significantly with flow and subjective well-being, but acceptance had a negative association with flow. Surprisingly, present-moment attention significantly and negatively correlated with experiential avoidance, while no significant correlations between awareness, acceptance and experiential avoidance were found. Taken together, the convergent and current validities of present-moment attention and awareness have been established, except for acceptance (Table 4).

Table 4

Means (M), standard deviations (SD) and Cronbach's coefficients (α) of all other measures, and Pearson's correlations between the subscales of the AMQ and other measures (Study 3).

Scale	M	SD	Cronbach's α	Subscales of the AMQ
				Present-moment attention	Awareness	Acceptance
MAAS	4.18	0.69	0.85	0.25**	0.13*	−0.06
AAQ-II	21.02	7.91	0.85	−0.26**	−0.10	0.00
SDFS	30.76	4.65	0.68	0.43**	0.30**	−0.16**
TCWS	24.88	6.65	0.70	0.37**	0.15**	−0.10

Abbreviations: AAQ-II = Acceptance and Action Questionnaire–II; AMQ = Athlete Mindfulness Questionnaire; MAAS = Mindful Attention Awareness Scale; SDFS = Short Dispositional Flow Scale; TCWS = Training and Competition Well-being Scale.

p < 0.05;

p < 0.01.

Study 4: cross-validating the factor structure of AMQ and obtaining additional validity evidence

The results of the CFA for the 12-item 3-factor solution indicated an excellent fit to the data: χ2(51) = 101.13, p < 0.001, CFI = 0.95, TLI = 0.94, WRMR = 0.86, RMSEA = 0.06 (90%CI: 0.041–0.074). Table 2 displays item means, standard deviations, standardized factor loadings and residuals, and Table 3 shows factor correlations and internal consistency estimates. In addition, a single-factor model was also tested, which produced a very poor fit to the data: χ2(54) = 259.89, p < 0.001, CFI = 0.81, TLI = 0.76, WRMR = 1.46, RMSEA = 0.11 (90%CI: 0.100–0.128).

Concurrent validity

The concurrent validity of present-moment attention was established, through the building of significant and negative associations among present-moment attention and burnout, negative affect and anxiety, and significant and positive associations among present-moment attention and vitality, positive affect, and enjoyment. Surprisingly, although a significant and positive correlation between awareness and positive affect was found, no significant correlations were revealed among awareness, acceptance and the other criteria-related variables (Table 5).

Table 5

Means (M), standard deviations (SD) and Cronbach's coefficients (α) of all other measures, and Pearson's correlations between the subscales of the AMQ and other measures (Study 4).

Scale	M	SD	Cronbach's α	Subscales of the AMQ
				Present-moment attention	Awareness	Acceptance
ABQ-RSA	13.28	3.50	0.73	−0.34**	−0.07	0.06
ABQ-EE	14.67	3.78	0.80	−0.16**	0.02	−0.03
ABQ-D	12.37	4.18	0.83	−0.27**	0.05	−0.00
SVS	28.65	7.73	0.89	0.37**	0.07	−0.01
PA	17.78	3.73	0.68	0.24**	0.12*	−0.04
NA	12.06	4.12	0.70	−0.20**	−0.03	0.02
SES	15.60	3.70	0.89	0.25**	0.01	−0.04
SCAT	18.61	3.42	0.71	−0.15**	−0.05	0.02

Abbreviations: ABQ = Athlete Burnout Questionnaire; AMQ = Athlete Mindfulness Questionnaire; D = devaluation; EE = emotional/physical exhaustion; NA = negative affect; PA = positive affect; RSA = reduced sense of accomplishment; SCAT = Sport Competition Anxiety Test; SES = Sport Enjoyment Subscale; SVS = Subjective Vitality Scale.

p < 0.05;

p < 0.01.

Development of a modified AMQ with direct-worded acceptance items

van Sonderen et al. demonstrated that reverse-worded items did not prevent response bias, but instead scores were contaminated by respondent inattention and confusion. Scores of reverse-worded acceptance items used in our study might have been contaminated and misinterpreted by athletes. We therefore changed the reverse-worded acceptance items into direct-worded items. Two reverse-worded acceptance items, with high factor loadings in Study 2, were changed into direct-worded items. Four direct-worded acceptance items, with high factor loadings in Study 2, were included. Two awareness items, with high factor loadings in Study 2, were also included. Accordingly, a 17-item modified AMQ was developed.

Study 5: validating the factor structure and providing validity evidence of the modified AMQ

Results of CFA on the 17-item 3-factor solution suggested a satisfactory fit to the data: χ2(116) = 267.54, p < 0.001, CFI = 0.93, TLI = 0.92, WRMR = 1.10, RMSEA = 0.06 (90%CI: 0.050–0.068), although Item 5 was found having a low factor loading (<0.30). After removing this item, model fit indices were significantly improved: χ2(101) = 221.28, p < 0.001, CFI = 0.95, TLI = 0.94, WRMR = 1.04, RMSEA = 10.06 (90%CI: 0.046–0.066). Table 6 displays item means, standard deviations, standardized factor loadings and residuals, and Table 7 shows factor correlations and internal consistency estimates. In addition, a single-factor model was also tested, and this produced a poor fit to the data: χ2(104) = 359.87, p < 0.001, CFI = 0.89, TLI = 0.87, WRMR = 1.35, RMSEA = 0.08 (90%CI: 0.070–0.088).

Table 6

Item means (M), standard deviations (SD), factor loadings (λ), and error variances (θ) of the AMQ (Study 5).

AMQ subscales and item content	M	SD	λ	θ
Present-moment attention
1. I can maintain my attention on my training.	3.93	0.82	0.65	0.58
4. When I find myself distracted, I gently bring my attention back to my training.	3.65	0.90	0.52	0.73
20. I can easily sustain my attention on the competition.	3.84	0.92	0.58	0.66
28. When I feel muscular pain during training, I can still maintain attention on things I should do.	3.66	0.93	0.62	0.62
33. If I notice that my mind is wandering, I can quickly get back to focusing on my training or competition.	3.56	0.90	0.69	0.52
Awareness
2. I am aware that my emotions during training and competition can influence my thinking and behavior.	3.68	0.92	0.57	0.67
14. When something unexpected happens during training or competition, I am aware of my emotion state.	3.60	0.87	0.64	0.59
21. When something during training and competition doesn't go well, I am aware of my inner frustration and restlessness.	3.71	0.85	0.41	0.83
24. When the situation changes during the competition, I am aware of the thoughts and ideas that flashed across my mind.	3.49	0.88	0.64	0.59
27. When the competition process is totally beyond my expectations, I am aware of my physical reactions and changes.	3.52	0.96	0.56	0.69
38. During training or competition, I can be immediately aware of my emotional changes.	3.62	0.93	0.71	0.50
Acceptance
3. During training and competition, I can put up with unpleasant thoughts and feelings.	3.69	0.88	0.70	0.52
7. During training and competition, it doesn't matter if the situation is good or bad, I can accept myself for who I am.	3.58	0.99	0.32	0.90
26. During training and competition, I can let go of the emotions brought about by negative life events.	3.57	0.90	0.57	0.68
29. During training and competition, it doesn't matter if my thoughts and feelings are comfortable or not, I put up with all of them.	3.46	0.91	0.44	0.81
31. Even though some thoughts and feelings during training and competition may be unpleasant or miserable, I can get along with them peacefully.	3.40	0.97	0.52	0.73

Note: All factor loadings are statistically significant at p < 0.01.

Abbreviation: AMQ = Athlete Mindfulness Questionnaire.

Table 7

Factor correlations and internal consistency reliabilities of the AMQ (Study 5).

	1	2	3
1. Present-moment attention	0.75	–	–
2. Awareness	0.48	0.76	–
3. Acceptance	0.56	0.38	0.64

Notes: All inter-factor correlations are statistically significant at p < 0.01. Composite reliability coefficients are present on the diagonal of the factor correlation matrix.

Abbreviation: AMQ = Athlete Mindfulness Questionnaire.

As predicted, convergent validity of present-moment attention, awareness, and acceptance was established, through establishing a significant and positive relationship with mindfulness as measured by the MAAS. Concurrent validities of the 3 mindfulness factors were also established via significant and negative associations with experiential avoidance, and burnout (reduced sense of accomplishment, emotional/physical exhaustion, and devaluation) and significant and positive associations with positive affect, dispositional flow, and well-being. Interestingly, no significant associations were revealed among present-moment attention, awareness, and negative affect, or between awareness and experiential avoidance (Table 8).

Table 8

Means (M), standard deviations (SD), and Cronbach's coefficients (α) of all other measures, and Pearson's correlations between the subscales of the AMQ and other measures (Study 5).

Scale	M	SD	Cronbach's α	Subscales of the AMQ
				Present-moment attention	Awareness	Acceptance
AAQ-II	21.41	8.09	0.86	−0.24**	−0.05	−0.26**
ABQ-RSA	13.75	3.27	0.65	−0.40**	−0.18**	−0.32**
ABQ-EE	13.11	4.14	0.84	−0.31**	−0.16**	−0.22**
ABQ-D	12.34	4.50	0.85	−0.32**	−0.23**	−0.23**
PA	17.02	4.23	0.71	0.38**	0.22**	0.17**
NA	11.84	4.30	0.75	−0.09	−0.01	−0.13*
MAAS	4.17	0.73	0.83	0.46**	0.21**	0.30**
SDFS	30.29	4.96	0.70	0.48**	0.38**	0.37**
TCWS	26.16	6.78	0.71	0.35**	0.13**	0.34**

Abbreviations: AAQ-II = Acceptance and Action Questionnaire–II; ABQ = Athlete Burnout Questionnaire; AMQ = Athlete Mindfulness Questionnaire; D = devaluation; EE = emotional/physical exhaustion; MAAS = Mindful Attention Awareness Scale; NA = negative affect; PA = positive affect; RSA = reduced sense of accomplishment; SDFS = Short Dispositional Flow Scale; TCWS = Training and Competition Well-being Scale.

p < 0.05; **p < 0.01.

Discussion

Along with the growing interest in applying mindfulness into a sport context, the development of psychometrically sound, sport-specific mindfulness scales is required. The purpose of the current study was to develop and validate a sport-specific mindfulness measure through 5 related studies using 4 separate samples of Chinese athletes. Based on the conceptualization of mindfulness in a sport context,6, 7 the AMQ consists of 3 subscales, namely, present-moment attention, awareness, and acceptance. In Study 1, evidence for content validity was provided. Although initial evidence of a 3-factor structure was established in Study 2 and further confirmed in Studies 3 and 4 using 2 independent samples, the convergent and concurrent validities of the acceptance subscale with reverse-worded items were not established in Studies 3 and 4. In Study 5, the factorial, convergent, and concurrent validities of the modified AMQ with direct-worded items were established, demonstrating that the AMQ is a reliable and valid measure of mindfulness in a sport context.

The 3 factors of mindfulness emerging from the current study are underpinned by the conceptual consideration of mindfulness in a sport context.6, 17 The psychometric evidence suggests that the AMQ adequately measures 3 key constituents of mindfulness in a sport context, namely, present-moment attention, awareness, and acceptance. Content validation by experts, coaches, and athletes has demonstrated high representativeness of the items as components of mindfulness in a sport context, and the construct validity of the AMQ was further supported in subsequent exploratory and confirmatory factor analyses. In addition to examining the 3-factor structure, a unidimensional structure was also tested; however, poor model fit was revealed consistently in Studies 3–5. This, therefore, may provide support for the proposition that mindfulness is a multidimensional construct that is represented by a number of separate but closely related factors, rather than 1 unidimensional factor. Nonetheless, further examination of the factor structure and construct validity of the AMQ is needed, given the fact that there are also many other general mindfulness instruments that have been developed in clinical and normal populations that may be applicable in a sport context.18, 44

The current study has revealed that direct-worded items instead of reverse-worded acceptance items may be more appropriate for athletes. Mathematically, problems may arise from the use of EFA in Study 2 in which both the direct-worded and reverse-worded acceptance items were generated as the majority of our candidate items are reverse-worded acceptance items. That is, EFA will generate 2 factors, instead of 1, if the pool of items is comprised of indicator items with 2 opposite directions of a single underlying factor, such as where one factor consists of direct-worded items and the other factor consists of reverse-worded items. The reason is that a linear factor analysis model cannot be fitted when the data are comprised of balanced positive and negative items of Likert-type scale. Conceptually, athletes might have misinterpreted the acceptance items in mindfulness that were in the reversed-worded format, given that the reverse-worded items contaminate scores rather than prevent response bias.43, 47 It is speculated that Chinese athletes may be more familiar and accustomed to the direct-worded items of acceptance and that this might align more closely with their experience (e.g., adversities and negative experiences). It is also speculated that athletes are normally trained to make judgments directly about experiences and ideas in a sport context. This suggests that individuals without meditation experience are more likely to make judgments whilst paying attention to present-moment experience.17, 44 Although athletes might agree with all statements of the items regardless of the content, findings from our study indicate that reverse-worded acceptance items cannot prevent response bias, but instead cause confusion.

Despite the lack of convergent and concurrent validities of the AMQ in Studies 3 and 4, the convergent and concurrent validities of a modified 16-item AMQ were established in Study 5 after converting the reverse-worded items into direct-worded items. Specifically, the convergent validity of the AMQ was established through building significant and positive relationships with mindfulness as measured by the MAAS. The concurrent validity was established through building significant associations with experiential avoidance, athlete burnout, and flow. The negative associations between mindfulness and experiential avoidance indicate that a higher level of mindfulness is related to a lower level of experiential avoidance, which is defined as an attempt to avoid unpleasant thoughts, images, feelings, sensations, and emotions. As experiential avoidance is an important component in mindfulness-based training for athletes,6, 7 it can be used to capture the changing process of mindfulness training. In addition, a negative association between mindfulness and athletes' burnout was revealed. Given the importance of burnout in the subjective well-being of athletes in training and competition, mindfulness training can be used to alleviate and prevent its occurrence. Furthermore, the positive associations between mindfulness and flow, revealed in the current study, are in line with previous findings that an increase in the state of mindfulness might facilitate athletes' flow experience in training and competition.9, 13, 17

From a practical perspective, the concept of mindfulness can be applied to performance enhancement in a competitive sport context as mindfulness is not a passive or dissociated state but a state where one is actively engaged in the observed experience. Emphasizing an accepting and nonjudgmental attitude is not to give up one's own judgments but rather to accept the nature of experience no matter whether they are good or bad. Thus, athletes should cultivate the capability of noticing, being aware of, and accepting both good and bad experiences (e.g., making an important decision) through mindfulness training in order to enhance performance or facilitate motor skill learning. Given that the direct-worded items of acceptance have emerged as being more appropriate to athletes, a direct attitude of acceptance should be emphasized instead of simply asking athletes not to make judgments on their private experience during mindfulness training. In addition, it is important to emphasize both the ability of sustaining attention and awareness in the present moment and the ability of refocusing to the present moment when athletes become distracted which is a typical phenomenon in sport context.

Although the AMQ with direct-worded items demonstrated psychometrically sound properties, several limitations should be noted and the findings should be taken with caution. Firstly, the questionnaires were developed using athletes without mindfulness experience, the AMQ can be further evaluated by athletes who have been trained in mindfulness. Secondly, given that the AMQ was developed using 4 separate samples of athletes from various sport training centers in the Mainland of China, caution should be taken when generalizing the findings to other contexts. Future studies could further validate the AMQ among Western athletes to examine its generalizability. Thirdly, the construct validity and internal consistency reliability have been confirmed using a cross-sectional design, but test–retest reliability and predictive validity have not been examined. Given the importance of examining the dynamic process of mindfulness training in athletes, future research should adopt longitudinal and intervention designs to better capture these 2 important indicators.

Conclusion

Through 5 related studies, the psychometrically sound AMQ which assesses mindfulness in a sport context in terms of 3 aspects: present-moment attention, awareness, and acceptance, has been developed and initially validated. Researchers and practitioners can use the AMQ to measure athletes' dispositional mindfulness to understand the effects of mindfulness training on adaptive and maladaptive psychological functioning. The importance of using direct-worded acceptance items to avoid confusion is also highlighted.

Authors' contributions

CQZ conceived of the study, conducted the data collection and data analysis, and drafted the manuscript under the supervision of PKC; GS has made a significant contribution to the study design, data collection, and manuscript preparation. All authors have read and approved the final version of the manuscript, and agree with the order of presentation of the authors.

Competing interests

The authors declare that they have no competing interests.