Reliability and Validity of the Chinese Version of the Cognitive Distortions Questionnaire (CD-Quest) in College Students.

BACKGROUND The aim of this study was to evaluate the reliability and validity of the Chinese version of the Cognitive Distortions Questionnaire (CD-Quest) among Chinese college students. MATERIAL AND METHODS A total of 460 college students from Jining were recruited in the study. Sample 1, including 239 college students, was tested for item analysis and exploratory factor analysis. Sample 2, including 221 college students, was tested for confirmatory factor analysis and criterion validity. The criterion validity was tested using the Negative Automatic Thoughts questionnaire (ATQ) and Dysfunctional Attitudes Scale (DAS). Test-retest reliability was evaluated in 40 college students from sample 1 with  4 weeks interval. RESULTS The corrected item-total correction (ITC) ranged from 0.675 (emotional reasoning) to 0.829 (unfair comparison). Exploratory factor analysis revealed the variance of the CD-Quest items was unidimensional, and it explained 55.261% of the data variance. Confirmatory factor analysis indicated all regression coefficients were higher than 0.4. The criterion validity was excellent, as shown by the relationships among CD-Quest, DAS, and ATQ (r=0.447, 0.566, respectively). Cronbach's alpha coefficient was 0.941 and the test-retest reliability was 0.928. CONCLUSIONS The Chinese version of CD-Quest had good reliability and validity, suggesting it is a reliable tool to evaluate cognitive distortion of Chinese college students.

Background

The concept of cognitive distortion, which is an important part of cognitive behavior theory, was first proposed by Beck. It refers to the interpretation and judgment of distortions and deviations of various situations and events from objective reality, which can lead to emotional distress and behavioral problems [1-5]. According to Beck’s cognitive theory of depression, cognitive distortion includes 3 levels of thoughts: self-schema of depression, maladaptive belief/hypothesis, and negative automatic thoughts [3]. Negative automatic thoughts and schema can affect a person’s way of interpreting problems, thus leading to inadequate coping strategies. Automatic thoughts refer to a series of cognitive products generated throughout the day, which are not the results of direct thinking or will. These thoughts usually contain negative views one has of himself, the world, or the future [6]. Schema is the core factor of Beck’s cognitive model. The core beliefs organized in self-schema can affect the development of beliefs, rules, and dysfunctional thoughts. The more reasonable cognitive distortions lead to a stronger emotional response [7,8]. Beck identified catastrophizing, emotional reasoning, extremes, labeling, and so on as common cognitive distortions, which could be evaluated and monitored along with unreasonable beliefs [4]. By identifying the distorted cognition, reconstruction of adaptive cognitive thoughts helps to adjust the individual’s mood and behavior. CBT is widely applied in clinical practice, including depression, anxiety, attention deficit disorder, drug dependence, obsessive-compulsive disorder, and gambling disorder [9-11]. More effective and convenient tools for assessing cognitive distortions are expected to be developed in the future.

In many cases, changes of thoughts, which are superficial, negative, and can be realized if taking a little effort, can be effective in improving depressive symptoms. These automatic thoughts are characterized by one or more thinking errors due to biased information processing. The negative automatic thoughts questionnaire (ATQ) developed by Hollon and Kendall [12] were used to assess the frequency of automatic thoughts. Other measures of cognitive distortions include the cognitive bias questionnaire (CBQ) and the cognitive error questionnaire [13,14]. The dysfunctional attitudes questionnaire was used to assess depression patients’ negative attitudes toward themselves, the external world, and the future [15]. The CD-Quest was developed by de Oliveira based on a review of books on cognitive therapy [16-18]. The questionnaire contained 15 cognitive distortion items that were reviewed and recommended by members of the Institute for Cognitive Therapy (). The contents of these cognitive distortions are superficial, automatic, and negative, similar to Beck’s negative automatic thoughts. The instructions of the CD-Quest included asking participants how often the cognitive error or distortion occurred in the past week (how often) and how strongly they believed in the idea at the moment the cognitive distortion occurred (but not now). The questionnaire was mentioned initially in the book by de Oliveira “Trial-based cognitive therapy: a manual for clinicians” [17]. It was originally designed as a clinical tool to help patients recall elements of cognitive errors or distortions made during the past week and to track their cognitive error and the subsequent link between emotional state and dysfunctional behavior, as well as to help clinicians and patients track a patient’s treatment progress. The questionnaire had good reliability and validity (Cronbach A=0.80~0.91, with a correlation coefficient of over 0.45 with anxiety, depression, and automatic thoughts) in samples of college students from Brazil [16], the United States [19], and Australia [20], the Turkish outpatient clinical sample [21], and the English version of social anxiety disorder [22]. At present, the scale is mainly used to evaluate changes in cognitive content, the relationship between cognitive content and symptoms, and changes in cognitive content before and after cognitive behavioral therapy [23-26].

The cognitive content assessment tools based on Beck’s cognitive theory in China mainly include the DAS, the ATQ, and the cognitive bias questionnaire (CBQ). The DAS mainly evaluates the dysfunctional cognitive contents [27]. The ATQ includes negative and extreme thoughts such as “I think it’s very difficult to live in the world” and “I can’t hold on” [28]. The CBQ is designed to measure the depression-related negative cognitive bias, and the questionnaire measures in 2 dimensions: 1) depression and 2) cognitive distortion [29]. However, these 3 questionnaires only reflect distorted cognition from one or several aspects. The CD-Quest, which is different from these 3 scales, contains 15 different types of distorted cognition, such as polarized thoughts and fate judgment (also known as catastrophizing). The questionnaire reflects the total degree of cognitive distortion and also tracks the degree of change of distorted cognition from the frequency and degree of cognitive distortion, so as to understand the change trend, extent of cognitive distortion, and types of distorted cognition, thus guiding treatment. The CD-Quest has been proved to have good psychometric characteristics in samples of college students in Brazil, the United States, Australia, and Turkey, and in a Turkish-speaking outpatient clinical sample [16,19-22].

At present, there is a lack of such questionnaires, which can be used to evaluate the severity of cognitive distortion through the total score, as well as reflect the degree and intensity changes of cognitive distortion in the treatment process in China. The application of CD-Quest in the Chinese cultural background needs further reliability and validity tests. Therefore, with the consent of de Oliveira, the purpose of this study was to explore the reliability and validity of the Chinese version of CD-Quest and to investigate its psychometric characteristics in the Chinese cultural background.

Material and Methods

Subjects

This was a cross-sectional survey study conducted between May 2017 and September 2017 that included 16 classes of college students from grade 1 to grade 4 in 2 colleges in Jining, Shandong Province, among which 20–30 students were randomly selected in each class and were investigated using the cluster random sampling method. With the consent of the dean in charge and the dean of the department of the 2 colleges, the selected college students accompanied by the counselors were asked to fill out the CD-Quest, ATQ, and DAS. The total test time was around 30 min. A total of 460 questionnaires were distributed. The questionnaires were completed in the student’s study room and collected on the spot, without invalid questionnaires. All participants signed the written informed consent before the study, and the relevant local ethics committee approved the design of the study.

Sample 1 included 239 students, including 99 males (41.6%) and 140 females (58.4%). The average age was 21.58±1.62, range 18–24 years. There were 35 psychiatric specialties and 204 clinical specialties used for reliability analysis, exploratory factor analysis, and criterion validity analysis.

Sample 2 was used for confirmatory factor analysis included 221 students, including 90 males (40.7%) and 131 females (59.3%); average age of 22.06±1.56 years, range 19–24 years; 33 were majoring in psychiatry and 188 were majoring in clinical medicine.

Forty students in random from sample 1 repeated the questionnaire after a 4-week interval, including 15 males (37.5%) and 25 females (62.5%), including 6 in psychiatry and 34 in clinical medicine. The average age was 21.98±1.57 years, range 19–24 years.

Measurement Tools

Cognitive distortions questionnaire (CD-Quest) [18]

This questionnaire was provided by the original author, de Oliveira, via email. When translating the CD-Quest, we followed the guidelines commonly used in cross-cultural research. First, the scale was translated into Chinese by the first author with the consent of scale developers provided via email. The translation of the scale was independently performed by 2 bilingual experts in the field, and all translations were compared with the original scale. After reviewing the original text and the translation, no items were added or deleted in the Chinese version. The final version was confirmed by consensus. The questionnaire consists of 15 items. The severity of cognitive distortion and the frequency and intensity of cognitive distortion including polarized thinking, fate judgment, underestimation of positive information, emotional reasoning, labeling, enlargement/reduction, selective abstraction, mind-reading, overgeneralization, personalization, arbitrary inference, blame (others or themselves), if… what should I do, and unfair comparisons.

The score in each column of each question represents the frequency of the idea, and the score in each row represents the extent to which the respondents believe in it. Frequency includes 4 options: (a) never; (b) occasionally (1–2 days), (c) most of the time (3–5 days), and (d) almost always (6–7 days). Similarly, in terms of intensity, respondents were asked how much they believed in their cognitive distortions and were given 4 choices: (a) not at all, (b) a little (up to 30%), (c) some (30–70%), and (d) very much (over 70%). Therefore, 3 different scores could be obtained from the questionnaire: (a) frequency score, (b) intensity score, and (c) total score. The scores for each item in the questionnaire were summed to get the total score (possible range=0–75).

Automatic thoughts questionnaire (ATQ) [28]

Holon and Kendall designed the automatic thoughts questionnaire in 1980 to evaluate the internal cognitive experience of depressed patients [12]. The Likert 5-grade scoring method was used, where 1 meant “never”, 2 “rarely”, 3 “sometimes”, 4 “often”, and 5 “always”. The higher score indicated more negative automatic thoughts. The score of ATQ ranged from 30 to 150. Cao et al. tested the reliability and validity of the scale [28], reporting that the Cronbach’s α coefficient was 0.9, and the scale had good structure validity, convergent validity, and differentiation validity.

Dysfunctional attitudes scale (DAS) [27]

The dysfunctional attitudes scale consisted of 40 items. A 7-point scale from “fully agree” to “completely disagree” was used for measuring the stable cognitive schema related to depression. The DAS showed good psychometric characteristics [30]. The split-half reliability of the Chinese version of DAS [27] was 0.835, and the Cronbach’s α coefficient was 0.87. The overall scale and each item group of the scale showed good validity.

Statistical analysis

IBM SPSS 24.0 was used for statistical analysis. The demographic information and clinical data of the subjects were analyzed using the normal distribution test and the descriptive statistical methods, including frequency, the mean±standard deviation, and the proportion of the scores.

The reliability analysis of the scale included Cronbach’s α analysis and item-total score correlation (the Cronbach’s alpha values considered to be acceptable were those exceeding 0.70). Test-retest reliability was calculated over a 4-week period.

Validity analysis included structure validity and criterion validity. The construct validity was assessed by using the exploratory principal component factor analysis, including principal component analysis (PCA) and variance maximum rotation method. Kaiser-Meyer-Olkin (KMO)>0.80 was considered as adequacy sampling. The significant Bartlett sphericity test was performed with the eigenvalue greater than 1. If the factor load was greater than or equal to 0.40, an intragroup correlation coefficient was used for the reliability analysis. After checking the eigenvalues and the scree plot, the parallel analysis was performed to select and extract factors.

In confirmatory factor analysis, Amos 17.0 was used for equation fitting test, and the fit indexes, including IFI (incremental fit index), CFI (comparative fit index), and NFI (normal fit index), were used. The model adaptation is considered good when the index is over 0.9. In Root Mean Square Error of Approximation (RMSEA), the model with the root means square of approximate error <0.08 is acceptable. If the minimum value of F (FMIN) is less than 2, the model is a good fit. All statistical tests were performed bilaterally (P<0.05).

Results

General demographic data

A total of 460 samples were collected, including 99 males (41.6%) and 140 females (58.4%) in 239 cases of group 1 and 90 males (40.7%) and 131 females (59.3%) in 221 cases of group 2, with age ranging from 19 to 24 years old and an average age of 22.06±1.56 years old. The gender difference between the 2 groups was not statistically significant. Demographic data of respondents are shown in Table 1.

Table 1

Demographic data of respondents.

	Male	Female	Total
Samples in group 1	99 (41.6%)	140 (58.4%)	239
Samples in group 2	90 (40.7%)	131 (59.3%)	221
	189 (41.09%)	271 (58.91%)	460

Reliability analysis

Reliability and test-retest reliability of internal consistency

The Cronbach’s α coefficient of CD-Quest was 0.941 in sample 1 and 0.918 in sample 2. The test-retest correlation coefficient with a 4-week interval was 0.928 among 40 college students in sample 1.

Item-total score correlation

Item-total correlation (ITC) ranged from 0.675 (emotional reasoning) to 0.829 (unfair comparison). The deletion of any item did not affect the increase of Cronbach’s α coefficient. ITC value and α value after item deletion are shown in Table 2.

Table 2

Analysis of psychological measurement characteristics of CD-Quest items in samples of college students (N=239).

	α if item deleted	ITC	Factor-loading	Communalities
Item 1	0.936	0.765**	0.766	0.587
Item 2	0.938	0.724**	0.723	0.523
Item 3	0.939	0.686**	0.675	0.456
Item 4	0.939	0.675**	0.672	0.452
Item 5	0.938	0.687**	0.691	0.478
Item 6	0.937	0.767**	0.762	0.580
Item 7	0.936	0.772**	0.764	0.583
Item 8	0.936	0.775**	0.778	0.606
Item 9	0.936	0.770**	0.774	0.599
Item 10	0.937	0.756**	0.758	0.575
Item 11	0.937	0.749**	0.739	0.546
Item 12	0.936	0.719**	0.760	0.578
Item 13	0.938	0.710**	0.713	0.508
Item 14	0.938	0.731**	0.729	0.531
Item 15	0.935	0.829**	0.828	0.686

ITC – corrected item-total correlation.

Validity analysis

Construct validity

Exploratory factor analysis

The results of exploratory factor analysis showed that the KMO value was 0.947, and the χ2 of Bartlett sphericity test was 2141.784 (df=105, P<0.001), which indicated that the sample data were suitable for exploratory factor analysis. The principal component analysis method was used for factor extraction, and the maximum variance orthogonal rotation was carried out. There was only 1 factor whose eigenvalue was >1 after rotation, which indicated that this was unidimensional. The commonality of items was 0.452–0.686, the cumulative contribution rate was 55.261% (eigenvalue=8.289), and a load of all items was 0.672–0.828 (Table 2). The scree plot also showed supportive evidence of a single-factor model (Figure 1).

Figure 1

The scree plot.

Confirmatory factor analysis

The exploratory factor analysis supported a unidimensional model. The goodness-of-fit of the factor model was verified by the confirmatory factor analysis. The results of fitting were CFI=0.925, IFI=0.926, NFI=0.889, FMIN=1.038, and RMSEA=0.078, all of which met the measurement requirements. All regression coefficients were ≥0.40. The structural equation model based on 15 regression coefficients is shown in Figure 2.

Figure 2

Confirmatory factor analysis of the CD-Quest.

Criterion-related validity

The score of each item and the total score of CD-Quest were all significantly correlated with the total scores of DAS and ATQ (P<0.01), and the correlation coefficient between the total score of CD-Quest and the total score of ATQ (r=0.566**) was higher than the total CD-Quest score and the total DAS score (r=0.447**) (P < 0.01). The Cronbach’s α coefficients of the internal consistency of ATQ, DAS were 0.975, 0.922 respectively. The relationship among CD-Quest, ATQ, and DAS is shown in Table 3.

Table 3

Correlation between CD-Quest items and ATQ, DAS.

	Total score of ATQ	Total score of DAS
Item 1	0.441**	0.382**
Item 2	0.451**	0.377**
Item 3	0.383**	0.192**
Item 4	0.372**	0.346**
Item 5	0.324**	0.276**
Item 6	0.471**	0.341**
Item 7	0.407**	0.263**
Item 8	0.389**	0.335**
Item 9	0.470**	0.401**
Item 10	0.476**	0.390**
Item 11	0.389**	0.318**
Item 12	0.332**	0.305**
Item 13	0.423**	0.353**
Item 14	0.398**	0.297**
Item 15	0.545**	0.428**
Total score of CD-Quest	0.566**	0.447**

p<0.01.

Discussion

According to Beck’s cognitive theory of depression, distorted cognition, unreasonable belief, or biased information processing are the important factors that can lead to depression. By identifying negative automatic thoughts or dysfunctional cognition and rebuilding these distorted thoughts or beliefs, it is possible to affect mood and behavior positively, thus relieving depression. Currently, the CDQ-Quest has been used in Brazil [16], the United States [19], Australia [20], Turkey [21], and other countries. However, its reliability and validity when used in college students in China has not been assessed. Because there is a lack of a questionnaire in China that can be used to evaluate cognitive distortion in relation to both degree and frequency, in the present study we used the CD-Quest with the consent of Professor de Oliveira, and tested its reliability and validity among Chinese college students so as to provide an effective evaluation tool and follow-up treatment for cognitive distortion among Chinese college students.

Results indicated that the Cronbach’s α coefficient obtained was significant for the total scale. All items were also significantly correlated with the total, demonstrating high conformity among them. The psychometric indicators of the Chinese version of the CD-Quest revealed that it was an adequate and promising measure of common cognitive distortions. This was consistent with the study results of the Australian general population by Kostoglou [21], the Turkish depression population by Sedat [21], and the anxiety population by Kaplan [22].

Exploratory factor analysis showed a single factor with a cumulative contribution rate of 55.261%, and the eigenvalue was 8.289. This result supports the results of de Oliveira, Kostoglou, and Morrison that CD-Quest was unidimensional [16,19,20].

Confirmatory factor analysis showed all fit indices of the unidimensional met the measurement requirements. All regression coefficients were ≥0.40, which indicates a good fit for each of the scale indicators. This meant that the individual indicators could well explain the scale construct and conformed to the scale design, thus suggesting that the Chinese version of the CD-Quest had good construct validity.

The correlation analysis between the total score of CD-Quest and dysfunctional attitude, and negative automatic thoughts showed a significant correlation, which suggested that CD-Quest could be used to evaluate the degree of cognitive distortion in content design, which agrees with the initial concept of scale design [20].

This study assessed the reliability and validity of the CD-Quest, and found that the cognitive distortion questionnaire has good reliability and validity among college students under the background of Chinese culture, which can be used as an effective measurement tool to evaluate the cognitive distortion of college students. Individual cognitive content by the development of the nervous system in the brain, life experience, social and cultural influence, so the individual self-consciousness and cognitive contents under different social and cultural background or environment may be obviously different.CD-Quest can be used to assess individual distorted cognitive thoughts and find differences among them. Assessment of distorted cognition can help patients with traumatic experiences improve their mood and behavior [31]. It is suggested that the evaluation of cognitive distortion should be further extended to different populations in the same culture and the same population in different cultures, so as to understand the influence of microgenetic factors on individual cognition [32]. The correlation between brain structure and function and cognitive distortion can also be explored by evaluating cognitive distortion.

The present study also has some limitations that need to be pointed out. Firstly, the samples were all from the same region, and the relative representativeness was poor. In future studies, the sample size needs to be expanded to make it more representative in China. Secondly, future clinical samples should be carefully selected in order to better serve the clinical practices and provide a better cognitive distortion assessment tool for the efficient evaluation of cognitive therapy. Thirdly, convergent and discriminant validity were evaluated only with self-report measures, which has subjective biases and are limited to information available exclusively by conscious awareness. Distorted beliefs about emotions or core beliefs about the self may exist but have not yet reached conscious awareness. Our study did not discuss the relationship between the CD-Quest and other known measures of cognitive distortions. Fourth, only negative biased cognitive content was evaluated. Lastly, the study did not explore how these distorted cognitions change as a function of treatment.

Conclusions

As predicted, the results were consistent with all previous studies [16,19-21]. Exploratory factor analysis and confirmatory factor analysis revealed that the CD-Quest has a unidimensional factor structure. Our results validate this structure in college students, supplying further evidence that 1 broad factor forms the base for the tendency to undergo the types of thinking patterns recognized in the CD-Quest. In summary, CD-Quest is a questionnaire that can be used to evaluate both the total score of cognitive distortion and the frequency and degree of cognitive distortion. To the best of our knowledge, this is the first study to evaluate the application of CD-Quest in Chinese students. Further research is needed to explore how these distorted cognition change as a function of treatment and the effect of microgenetic factors on individual cognition.