Assessing Negative Automatic Thoughts: Psychometric Properties of the Turkish Version of the Cognition Checklist.

BACKGROUND: Beck's theory of emotional disorder suggests that negative automatic thoughts (NATs) and the underlying schemata affect one's way of interpreting situations and result in maladaptive coping strategies. Depending on their content and meaning, NATs are associated with specific emotions, and since they are usually quite brief, patients are often more aware of the emotion they feel. This relationship between cognition and emotion, therefore, is thought to form the background of the cognitive content specificity hypothesis. Researchers focusing on this hypothesis have suggested that instruments like the cognition checklist (CCL) might be an alternative to make a diagnostic distinction between depression and anxiety.
OBJECTIVES: The aim of the present study was to assess the psychometric properties of the Turkish version of the CCL in a psychiatric outpatient sample. PATIENTS AND METHODS: A total of 425 psychiatric outpatients 18 years of age and older were recruited. After a structured diagnostic interview, the participants completed the hospital anxiety depression scale (HADS), the automatic thoughts questionnaire (ATQ), and the CCL. An exploratory factor analysis was performed, followed by an oblique rotation. The internal consistency, test-retest reliability, and concurrent and discriminant validity analyses were undertaken.
RESULTS: The internal consistency of the CCL was excellent (Cronbach's α = 0.95). The test-retest correlation coefficients were satisfactory (r = 0.80, P < 0.001 for CCL-D, and r = 0.79, P < 0.001 for CCL-A). The exploratory factor analysis revealed that a two-factor solution best fit the data. This bidimensional factor structure explained 51.27 % of the variance of the scale. The first factor consisted of items related to anxious cognitions, and the second factor of depressive cognitions. The CCL subscales significantly correlated with the ATQ (rs 0.44 for the CCL-D, and 0.32 for the CCL-A) as well as the other measures of mood severity (all Ps < 0.01). To a great extent, all items of the CCL were able to distinguish the clinical and non-clinical groups, suggesting the scale has high discriminating validity.
CONCLUSIONS: The current study has provided evidence that the Turkish version of the CCL is a reliable and valid instrument to assess NATs in a clinical outpatient sample.

1. Background

The cognitive theory of emotional disorders developed by Beck (1) has been enormously influential in psychiatry (2-4). Hypotheses about the relationship between cognition and emotion that were derived from this theory have led to a greater understanding of many psychopathological states, and to an effective treatment modality, i.e., cognitive behavioral therapy, which has impressively shaped the psychiatry literature ever since (5-8). Beck’s cognitive theory posits that mood states may be discriminated on the basis of their unique cognitive contents (1, 9). According to the cognitive content specificity hypothesis, automatic thoughts and subjectively perceived emotional states should be positively related (10-12).

Beck’s theory of emotional disorder suggests that negative automatic thoughts (NATs) and the underlying schemata affect one’s way of interpreting situations and result in maladaptive coping strategies (1, 13). This theory proposes a three-layer cognitive structure, where NATs are topographically located on the outermost surface. NATs are the most easily accessible cognitions, and they tend to be the easiest to work on with patients during therapy sessions. NATs distort reality, are emotionally distressing, and interfere with patients’ functionality. Depending on their content and meaning, NATs are associated with specific emotions, and since they are usually quite brief, patients are often more aware of the emotions they feel (13). This relationship between cognition and emotion, therefore, is thought to form the background of the cognitive content specificity hypothesis.

Before cognitive variables were described in a taxonomy by Kendall and Ingram (14), there was confusion about the use of the term cognitive content specificity. Yet, now the term is more widely accepted as “a more specific claim that certain themes of semantic content in self-reported automatic thoughts are unique to either depression or anxiety” (15). Although other theories have attempted to explain the cognitive content specificity, e.g., the self-discrepancy theory by Higgins (16), the most investigated formulation to date has been the hypothesis by Beck (9). After the development of an assessment tool to discriminate between depressive and anxious cognitions, i.e., the cognition checklist (CCL) (10), research aiming to improve the differential diagnosis of mood and anxiety disorders has increased substantially (12, 15, 17-25). Researchers focusing on cognitive content specificity have suggested that instruments like the CCL might be an alternative to make a diagnostic distinction between depression and anxiety. This suggestion stemmed from clinical observations demonstrating that depressive cognitive content was focused on themes related to negative self-evaluation, hopelessness, and pessimism about the future (9, 26, 27), whereas anxious cognitive content was more focused on physical or psychological threat, and an inability to cope with danger (26, 28). Another way of differentiating depressive and anxious cognitive content was proposed to be associated with the temporal focus of cognitions, where depressive cognitions were more likely to be past-oriented, and anxious cognitions future-oriented (29).

A meta-analysis on the cognitive content specificity hypothesis concluded that the hypothesis was only supported for depressive cognitive content (15). The authors argued that anxious cognitive content’s poor performance in demonstrating specificity might be due to the possibility that (i) anxious cognitive content might not be specific to anxiety and (ii) the themes involved in anxious cognitive content might be shared variables between depression and anxiety. This is further supported by studies using prototypical cognitions to distinguish different diagnostic categories (18, 30) and on positive and negative affectivity (24, 26, 31, 32).

Currently available measures to assess negative cognitions in depression or anxiety for adults are limited. In addition to the CCL (10), the literature review provided us with the following measures: (i) the automatic thoughts questionnaire (ATQ) (33), (ii) the Crandell cognitions inventory (CCI) (34), (iii) the anxious self-statements questionnaire (ASSQ) (35), (iv) the UBC cognitions inventory (UBC-CI) (30), (v) the agoraphobic cognitions questionnaire (ACQ) (36), and (vi) the body sensations questionnaire (BSQ) (36). These questionnaires, except for the ATQ, the CCI, and the depression subscale in the UBC-CI, focus on thematically related cognitions to anxiety, whereas the former questionnaires focus solely on depressive cognitions. Apart from the UBC-CI, which has both subscales for depression and anxiety, the CCL is, therefore, unique in that it consists of two different subscales focused on depressive or anxious cognitions. The anxiety subscale (CCL-A) has less prototypical items in the sense that they do not particularly represent core features of the aspects of specific anxiety disorders. Therefore, these items may be classified as general in terms of anxiety related cognitions, i.e., future-oriented threat. The UBC-IC, however, consists of more prototypical and disorder-specific anxiety subscales, e.g., worry, panic, somatic preoccupation, and social fear. This distinction might position the CCL-A as a more transdiagnostic perspective on anxiety, whereas the specific anxiety subscales of the UBC-IC might be more relevant for research involving disorder-specific approaches to anxiety. The depression subscale of the CCL (CCL-D), as reported in the literature (15), may also be conceptualized as disorder specific in the sense that its items reflect the core aspects of depression.

The CCL was initially developed to differentiate anxiety and depression and to measure the frequency of automatic thoughts (10). It was initially thought to explicitly test the cognitive content specificity hypothesis of the cognitive model (1, 27). Its psychometric properties indicate that it is a reliable and valid tool. Cronbach’s α values range from 0.90 - 0.91 to 0.92 - 0.93 in psychiatric outpatients and 0.86 and 0.90 in students, for the CCL-A and CCL-D, respectively. It has also been shown to have high test-retest reliability and concurrent validity with scales measuring depression and anxiety severity (10, 37). The subscales have also demonstrated evidence for discriminant validity, differentiating patients diagnosed with depression or anxiety (10, 37). The results have also indicated that the CCL-D and CCL-A are moderately correlated with each other, which might be due to the shared variance between the subscales (37). Factor analytic studies have generally revealed that the CCL consists of two subscales, which correspond to depression and anxiety related negative cognitions (10, 37, 38), although some findings differ (19). Apart from these initial studies, later research has also consistently reported that the CCL is a reliable and valid tool for research purposes (39-41).

2. Objectives

The aim of the present study was to assess the psychometric properties of the Turkish version of the CCL in a psychiatric outpatient sample. The hypotheses were that the CCL subscales would correlate significantly with measures of depressive and anxiety symptoms and a similar scale measuring NATs and the CCL subscales could distinguish between clinical diagnostic subgroups.

3. Patients and Methods

3.1. Participants

A total of 425 psychiatric outpatients 18 years of age and older presenting to one secondary and two tertiary healthcare services in two different cities were recruited. Participants were excluded from the study if they (i) were diagnosed with psychotic disorders, bipolar mood disorders, organic mental disorders, substance use disorder, dementia, and/or mental retardation, (ii) suffered from a medical/neurologic disorder not currently under control, (iii) were suicidal at the time of the intake interview, and (iv) had a history of head trauma, brain surgery, or electroconvulsive therapy.

3.2. Assessment Instruments

Demographic and Clinical Data Form: This form was developed by the researchers and the demographic data, i.e., age, gender, level of education, marital status, occupation status, and clinical variables, i.e., primary psychiatric diagnosis, family history of mental disorder, comorbid medical disorder, were recorded onto it.

Mini International Neuropsychiatric Interview (MINI) (42): The MINI is a structured clinical diagnostic interview for mental disorders. In the present study, all participants were diagnosed by experienced clinicians according to the Diagnostic and Statistical Manual of Mental Disorders version IV (DSM-IV) (43) with the Turkish version of the MINI (44).

Cognition checklist (CCL) (10): The CCL is a 26-item 5-point self-report scale that is used to assess the frequency of NATs reported by patients (0 = never, 4 = always), and it has two subscales, a 14-item depression subscale (CCL-D), characterized by hopelessness, generally pessimistic assessments of the world and the future, and negative self-judgment, and a 12-item anxiety subscale (CCL-A), characterized by general future-oriented physical or psychological threat, and danger (15, 30). The respondents are required to rate the frequency of their depressive and anxious automatic thoughts across a variety of situations, e.g., while feeling pain or physical discomfort, attending a social situation, working on a project, with a friend (possible ranges 0 - 56 for CCL-D, and 0 - 48 for CCL-A). For the present study, the respondents were asked to rate their thoughts regardless of the situation. For the translation of the CCL, guidelines widely used in cross-cultural research were followed (45, 46). First, the developer of the scale was contacted by e-mail, and after his approval, the scale was translated into Turkish by the first author of this article. The translated scale was independently back-translated by two bilingual experts in the field, and all translations were compared with the original scale. After reviewing the original and translated versions, a final version of consensus was adopted.

Automatic thoughts questionnaire-negative (ATQ) (33): The ATQ is a 30-item 5-point self-report scale that assesses the frequency of NATs. For each item, respondents are asked to indicate how frequently each thought occurred during the past week (1 = not at all, 5 = all the time). The ATQ predominantly concentrates on negative cognitions characteristic of depression, e.g., loss, failure, self-depreciation (10) and consists of statements that reflect different aspects of depression, e.g., demoralization, self-criticism, brooding, amotivation, and interpersonal disappointment (47). The ATQ has been reported to have excellent psychometric properties and to differentiate between depressed and non-depressed groups (48, 49). The Turkish version of the ATQ, which has good reliability (Cronbach’s α = 0.93) and validity, was used (50). Only the total score of the ATQ was used in the analyses (possible range = 30 - 150).

Hospital anxiety and depression scale (HADS) (51): The HADS consists of 14 items, divided into two equal subscales of depression and anxiety. It is a 4-point self-report instrument, and cut-off scores of 7 for depression, and 10 for anxiety have been proposed. The Turkish version of the HADS (used in the present study) is a reliable (Cronbach’s α = 0.78 for depression, and 0.85 for anxiety) and valid instrument (52). Both subscale scores were calculated for analyses.

3.3. Procedure

The diagnostic interview was administered face-to-face at intake by trained psychiatrists. Participants completed the self-report measures after the intake interview. All questionnaires were administered in random order. It took about 20 - 30 minutes for all the questionnaires to be completed. No compensation of any sort was offered. All participants signed a written informed consent before the study, and the respective local ethics committees approved the study design.

3.4. Statistical Analyses

All analyses were performed using IBM SPSS for Windows, Version 22.0 (53). Participants’ demographic and clinical data were analyzed by descriptive statistics. For group comparisons, Student’s t-test was applied. An exploratory factor analysis (principal axis factoring) was performed, followed by an oblique rotation (direct oblimin). Factors for extraction were selected by examining eigenvalues (54), and the scree plot, and by conducting a parallel analysis (55-58). To assess the internal consistency of the scale, Cronbach’s α was computed. The correlation coefficient was calculated for test-retest reliability over a 4-week period. For concurrent validity, bivariate Pearson correlation analyses were conducted with the measures of depression and anxiety severity, and of NATs. The correlation coefficients were compared with each other for statistical significance by Steiger’s Z test (59). For discriminant validity, partial correlation coefficients between the same- and opposite-affect rating scales and the CCL subscales were calculated. Statistical significance was set at a P-value of < 0.05.

4. Results

4.1. Descriptive Statistics

A total of 425 psychiatric outpatients 18 years of age and older (60.9 % female; mean age = 37.9 years, SD = 12.76, range = 18 - 65) were recruited for the study. Almost two thirds of the participants (n = 275, 64.7 %) were married, and 31.3 % of them (n = 133) were single. Almost all the participants (n = 382, 89.9%) were at least graduates of high school, and 54.6 % (n = 232) had a job with a regular income.

The primary DSM-IV diagnoses of the participants were as follows: 32.24 % (n = 137) depressive disorders (e.g., major depressive disorder, dysthymia) and 28.94 % (n = 123) anxiety disorders (e.g., generalized anxiety disorder, panic disorder, social anxiety disorder, obsessive-compulsive disorder, posttraumatic stress disorder). Over one third of the participants had a comorbid psychiatric diagnosis, the most common being a comorbid depressive and anxiety disorder (37.41 %, n = 159). Seventy-one of the participants (16.7 %) reported that they had a family member diagnosed with some kind of mental disorder, and 119 of the participants (28 %) were also suffering from a comorbid medical condition.

The mean scores and standard deviations of the individual items and the total score of the CCL subscales as well as the other scales used in the current study are presented in Table 1.

Table 1.

Descriptive Statistics of the Individual Items and the Total Score of the Subscales of the CCL, the ATQ, and the Scales of Mood Severity

	Values[a]	Minimum	Maximum	Possible Range
HADS-Dep	7.11 ± 4.80	0	21	0 - 21
HADS-Anx	9.55 ± 4.81	0	21	0 - 21
ATQ-Total	61.01 ± 25.98	34	120	30 - 150
CCL-D	5.59 ± 6.03	0	24	0 - 56
CCL-A	6.97 ± 8.13	0	34	0 - 48
Item 1	0.50 ± 0.66	0	4	0 - 4
Item 2	0.36 ± 0.57	0	4	0 - 4
Item 3	0.52 ± 0.77	0	4	0 - 4
Item 4	0.96 ± 1.01	0	4	0 - 4
Item 5	0.26 ± 0.51	0	4	0 - 4
Item 6	0.24 ± 0.49	0	4	0 - 4
Item 7	0.38 ± 0.57	0	4	0 - 4
Item 8	0.29 ± 0.60	0	4	0 - 4
Item 9	0.25 ± 0.51	0	4	0 - 4
Item 10	0.29 ± 0.60	0	4	0 - 4
Item 11	0.25 ± 0.50	0	4	0 - 4
Item 12	0.26 ± 0.62	0	4	0 - 4
Item 13	0.38 ± 0.76	0	4	0 - 4
Item 14	0.65 ± 0.86	0	4	0 - 4
Item 15	0.79 ± 0.92	0	4	0 - 4
Item 16	0.43 ± 0.75	0	4	0 - 4
Item 17	0.43 ± 0.71	0	4	0 - 4
Item 18	0.43 ± 0.74	0	4	0 - 4
Item 19	0.72 ± 0.97	0	4	0 - 4
Item 20	0.60 ± 0.98	0	4	0 - 4
Item 21	0.64 ± 0.96	0	4	0 - 4
Item 22	0.51 ± 0.77	0	4	0 - 4
Item 23	0.48 ± 0.86	0	4	0 - 4
Item 24	0.62 ± 0.92	0	4	0 - 4
Item 25	0.89 ± 1.10	0	4	0 - 4
Item 26	0.43 ± 0.81	0	4	0 - 4

aData are presented as mean ± SD.

4.2. Internal Consistency and Split-Half Reliability

The internal consistency of the CCL was excellent (Cronbach’s α = 0.95). The corrected item-total correlation (ITC) coefficients ranged from 0.41 (“I am a social failure.”) to 0.81 (“Something awful is going to happen.”). Deletion of none of the items resulted in an increase in the Cronbach’s α value of the scale. The ITC values are shown in Table 2.

Table 2.

Factor Loadings After the Exploratory Factor Analysis, and the Corrected Item-Total Correlations of the CCL

	Factor 1	Factor 2	Communalities	ITC
Item 1	0.21	0.32	0.23	0.47
Item 2	0.05	0.72	0.56	0.64
Item 3	0.01	0.55	0.31	0.48
Item 4	0.09	0.59	0.29	0.41
Item 5	0.12	0.63	0.50	0.62
Item 6	0.10	0.71	0.43	0.49
Item 7	0.04	0.73	0.57	0.64
Item 8	0.18	0.39	0.27	0.48
Item 9	0.09	0.80	0.56	0.56
Item 10	0.24	0.43	0.36	0.56
Item 11	0.15	0.65	0.55	0.64
Item 12	0.28	0.58	0.60	0.71
Item 13	0.40	0.49	0.63	0.75
Item 14	0.50	0.20	0.41	0.63
Item 15	0.67	0.05	0.49	0.66
Item 16	0.73	0.05	0.58	0.69
Item 17	0.71	0.09	0.58	0.69
Item 18	0.60	0.21	0.55	0.70
Item 19	0.66	0.14	0.57	0.73
Item 20	0.97	0.33	0.84	0.69
Item 21	0.75	0.06	0.62	0.74
Item 22	0.52	0.29	0.54	0.70
Item 23	0.62	0.21	0.58	0.74
Item 24	0.75	0.15	0.71	0.81
Item 25	0.80	0.11	0.55	0.63
Item 26	0.37	0.40	0.47	0.67

Abbreviation: ITC: corrected item-total correlation.

The test items were split in half, where both halves consisted of 13 items, and both halves correlated with each other (r = 0.71). The Cronbach’s α values were 0.89 for part 1 and 0.94 for part 2. The Spearman-Brown split-half reliability coefficient was good (0.83).

The internal consistencies of the CCL depression and anxiety subscales were excellent as well (Cronbach’s αs 0.90 and 0.94, respectively). The ITC coefficients ranged from 0.47 to 0.71 for the CCL-D, and from 0.63 to 0.78 for the CCL-A. Deletion of none of the items resulted in an increase in the Cronbach’s α values of the subscales.

4.3. Test-Retest Reliability

For a subgroup of patients (n = 70, 16.47 %), the test-retest correlation coefficient was calculated over a 4-week period. The results were satisfactory (r = 0.80, p < 0.001 for CCL-D, and r = 0.79, P < 0.001 for CCL-A). The test-retest reliabilities of the subscales did not differ.

4.4. Exploratory Factor Analysis

The Kaiser-Meyer-Olkin measure was 0.84, and the Bartlett’s test of sphericity was highly significant (χ2 = 10716.83, P < 0.001). An exploratory factor analysis, followed by an oblique rotation, revealed that a two-factor solution best fit the data. This bidimensional factor structure explained 51.27 % of the variance of the scale (eigenvalues 11.45, and 1.88, respectively). The first factor consisted of items related to anxious cognitions, and the second factor of depressive cognitions. Factor loadings of the items are shown in Table 2.

The factor loadings did not differ from the original scale, except for items 14 (“I have become physically unattractive.”) and 26 (“I’m losing my mind.”), which loaded on the opposite-affect factor more strongly. Items that cross-loaded on both factors (strength of loading difference on different factors < 0.30) were items 1 (“I’m worthless.”), 8 (“I’ve lost the only friends I’ve had.”), 10 (“I’m worse off than they are.”), 13 (“Nothing ever works out for me anymore.”), and 26. The only items that failed to load > 0.40 on their corresponding factors were items 1, 8, 14, and 26. Yet, all of these items were left in their original subscale, as they thematically corresponded more appropriately to their same-affect factor.

4.5. Concurrent Validity

Table 3 presents the correlations between the CCL subscales and other measures used in the study. As hypothesized, the CCL subscales significantly correlated with the ATQ (rs 0.44 for the CCL-D, and 0.32 for the CCL-A) as well as the other mood severity measures (all Ps < 0.01). As expected, the CCL-D correlated more strongly with the HADS-D (r = 0.43), and the CCL-A with the HADS-A (r = 0.42). However, these correlations with the mood severity ratings were not statistically significantly different (Zs 1.64 for the comparison of correlation coefficients between the CCL-D and the same- and opposite-mood severity scales, and 0.83 for the comparison of correlation coefficients between the CCL-A and the same- and opposite-mood severity scales). The stronger correlation between the CCL-D and the ATQ than between the CCL-A and the ATQ was also expected, as the ATQ focuses predominantly on depressive cognitions, and the former correlation was statistically significantly different from the latter (Z = 1.97, P < 0.05). These results demonstrate that there is concurrent validity of the scale.

Table 3.

Bivariate and Partial Correlations Between the CCL Subscales and the Other Measures

Correlation coefficients	HADS-Dep	HADS-Anx	ATQ
CCL-D
r	.43[a]	.33[a]	.44[a]
Partial r	.30[b]	.17	.37[a]
CCL-A
r	.37[a]	.42[a]	.32[a]
Partial r	.09	.24[b]	.17[b]

aP < 0.01.

bP < 0.05.

4.6. Discriminant Validity

Participants were divided into four groups according to the cut-off scores of the HADS subscales (depressed / non-depressed, and anxious / non-anxious), and the mean scores of the individual items and the total score of the CCL subscales were compared between these groups. Except for item 8 among the participants with anxiety, all items were able to distinguish the groups, suggesting that the scale has high discriminating validity (all Ps < 0.05). The results are shown in Table 4.

Table 4.

Discriminating Between Clinical and Non-Clinical Participants According to the CCL Scores[a]

CCL	HADS-Dep			HADS-Anx
	≥ 7 (n = 217)	< 7 (n = 208)	t	≥ 10 (n = 202)	< 10 (n = 223)	t
Item 1	0.58 ± 0.66	0.41 ± 0.65	2.632[b]	0.58 ± 0.67	0.42 ± 0.65	2.556[c]
Item 2	0.43 ± 0.58	0.28 ± 0.56	2.794[b]	0.42 ± 0.60	0.30 ± 0.54	2.168[c]
Item 3	0.65 ± 0.83	0.38 ± 0.69	3.643[b]	0.63 ± 0.83	0.43 ± 0.71	2.716[b]
Item 4	1.15 ± 1.02	0.77 ± 0.96	3.882[b]	1.09 ± 1.03	0.85 ± 0.97	2.535[c]
Item 5	0.39 ± 0.61	0.13 ± 0.34	5.464[b]	0.35 ± 0.56	0.19 ± 0.45	3.226[b]
Item 6	0.31 ± 0.54	0.16 ± 0.42	3.305[b]	0.32 ± 0.55	0.17 ± 0.42	3.218[b]
Item 7	0.54 ± 0.62	0.20 ± 0.44	6.524[b]	0.51 ± 0.62	0.26 ± 0.49	4.742[b]
Item 8	0.36 ± 0.60	0.21 ± 0.58	2.738[b]	0.34 ± 0.55	0.24 ± 0.63	1.798[d]
Item 9	0.42 ± 0.62	0.07 ± 0.25	7.620[b]	0.38 ± 0.62	0.13 ± 0.34	5.146[c]
Item 10	0.40 ± 0.67	0.18 ± 0.49	3.801[b]	0.41 ± 0.68	0.18 ± 0.50	4.033[b]
Item 11	0.37 ± 0.57	0.13 ± 0.38	5.063[b]	0.35 ± 0.57	0.16 ± 0.40	3.980[b]
Item 12	0.38 ± 0.77	0.13 ± 0.37	4.342[b]	0.37 ± 0.76	0.16 ± 0.44	3.501[b]
Item 13	0.59 ± 0.93	0.16 ± 0.40	6.130[b]	0.57 ± 0.91	0.21 ± 0.52	5.090[b]
Item 14	0.83 ± 0.93	0.47 ± 0.74	4.517[b]	0.86 ± 0.92	0.47 ± 0.75	4.865[b]
Item 15	0.99 ± 1.03	0.58 ± 0.73	4.766[b]	1.04 ± 1.01	0.56 ± 0.76	5.680[b]
Item 16	0.66 ± 0.88	0.19 ± 0.48	6.864[b]	0.65 ± 0.88	0.23 ± 0.55	5.903[b]
Item 17	0.59 ± 0.83	0.26 ± 0.52	4.818[b]	0.59 ± 0.82	0.28 ± 0.57	4.597[b]
Item 18	0.65 ± 0.89	0.19 ± 0.44	6.849[b]	0.58 ± 0.86	0.29 ± 0.58	4.145[b]
Item 19	0.96 ± 1.04	0.46 ± 0.83	5.430[b]	1.01 ± 1.15	0.45 ± 0.68	6.188[b]
Item 20	0.80 ± 1.05	0.40 ± 0.88	4.268[b]	0.93 ± 1.16	0.31 ± 0.66	6.807[b]
Item 21	0.84 ± 1.00	0.43 ± 0.87	4.526[b]	0.86 ± 1.10	0.45 ± 0.75	4.491[b]
Item 22	0.60 ± 0.86	0.41 ± 0.66	2.551[c]	0.61 ± 0.86	0.41 ± 0.67	2.757[b]
Item 23	0.73 ± 1.04	0.23 ± 0.49	6.323[b]	0.66 ± 1.03	0.32 ± 0.62	4.235[b]
Item 24	0.87 ± 1.08	0.37 ± 0.61	5.792[b]	0.91 ± 1.10	0.37 ± 0.62	6.321[b]
Item 25	1.06 ± 1.07	0.72 ± 1.10	3.371[b]	1.23 ± 1.18	0.59 ± 0.88	6.351[b]
Item 26	0.64 ± 0.96	0.21 ± 0.54	5.724[b]	0.62 ± 0.96	0.26 ± 0.59	4.732[b]
CCL-D	7.42 ± 6.87	3.67 ± 4.25	6.724[b]	7.18 ± 6.85	4.14 ± 4.76	5.344[b]
CCL-A	9.40 ± 9.38	4.44 ± 5.55	6.597[b]	9.69 ± 9.28	4.51 ± 5.94	6.918[b]

aData are presented as mean ± SD.

bP < 0.05.

cP < 0.01.

dP = 0.07.

The strong correlation between the subscales of the CCL (r = 0.73) also shows that there is substantial overlap in variance. Therefore, we also calculated partial correlations between the CCL subscale and one of the mood severity rating scales. After controlling for the remaining scale, partial correlations between each CCL subscale and the same-affect rating scale remained significant (Ps < 0.05), whereas the correlations between the opposite-affect rating scales became non-significant. Further, the partial correlations with the mood severity were only statistically significantly different for the depression subscale (Z = 3.15, P < 0.01). The partial correlations are shown in Table 3.

5. Discussion

NATs are hypothesized to be central in the development and treatment of psychopathology according to the theory by Beck (1, 13, 27). Yet, there is a limited number of scales that can be utilized to assess NATs in Turkish (50). Moreover, with the available scales it is not possible to assess for both depressive and anxious negative cognitions simultaneously. Therefore, the current study aimed to evaluate the psychometric properties of the Turkish version of the CCL, and the results in a psychiatric outpatient sample revealed that the scale had excellent internal consistency and good test-retest reliability, that it consisted of two discrete factors, and that it provided evidence for concurrent and discriminant validity.

The two-factor structure of the CCL is in line with the previous reports of the psychometric properties of the CCL. To a great extent, factor loadings of the individual items corresponded with the expected subscale of the CCL. The two items that loaded more strongly on the opposite-affect factor were decidedly left on their original subscales, as they were more relevant to these subscales according to their themes, i.e., item 14 and negative self-evaluation, item 26 and physical threat. Since five items of the CCL cross-loaded on both subscales, after the investigation of the content of these items, it was decided that they should be classified into the originally proposed subscales. The four items that failed to load on their respective subscales fell only quite circumstantially shorter than the expected values. Yet, items 1, 8, 14, and 26 were the most frequently problematic items, as listed above. Therefore, further investigation of these items in subsequent studies is warranted.

The CCL subscales were found to correlate positively with the measures of mood severity and the ATQ. First, the subscales’ association with the ATQ demonstrated that the CCL was able to successfully assess the same construct. Second, although the CCL subscales failed to selectively correlate with the same-affect measures of severity, the significant difference in the partial correlation between the CCL-D and the HADS depression subscale, when compared with the correlation between the CCL-A and the HADS anxiety subscale, provided further evidence that the CCL had both concurrent and discriminant validity. The latter was also substantially validated by the group comparisons, where the participants were classified according to cut-off scores of depression and anxiety.

The stronger and statistically different correlation between the CCL-D and the ATQ compared to the correlation between the CCL-A and the ATQ might be interpreted as a finding for the highly prototypical cognitive content of the depression items, whereas as previously reported in the literature (30), the anxiety items of the CCL might be suggestive of more general, less prototypical cognitive content. This finding might have some clinical implications, such that patients with depression and comorbid anxiety disorders would score high on both subscales (26, 40, 41, 60). On the other hand, depressed patients with no comorbid anxiety would score lower on the anxiety items of the CCL. This might be indicative for a differential diagnosis of depressed patients with and without anxiety disorders, as suggested by Beck (10, 37). Yet, the high scores on the anxiety subscale would not permit the diagnosis of a specific anxiety disorder, and further evaluation with a highly prototypical cognitive content for anxiety would be necessary, directed by the clinical presentation of the patients.

The strong correlation between the subscales of the CCL indicates that there is substantial overlap in the items for assessing negative cognitive content related to depression and anxiety. Although the cognitive content specificity hypothesis posits that NATs may enhance the probability of correct differential diagnosis (10, 37), results of this study have only partially supported this claim, which is in accordance with the literature (15). However, the main focus of the current study was not to evaluate the cognitive content specificity hypothesis, which would require additional assessments.

The current study is the first to provide evidence that the CCL has sound psychometric properties in a Turkish outpatient sample. Further, the CCL is the second scale available to assess negative depressive cognitions in Turkish; with its more comprehensive structure, which also allows for the assessment of anxiety related NATs, it is likely that it might provoke more research ideas focusing on the effect of NATs in the differential diagnosis, development, maintenance, and treatment of mental disorders.

The limitations of the study may be summarized as follows: (i) the limited number of participants, (ii) the limited number of psychopathology scales, (iii) the participants’ low scores on the depression and anxiety rating scales, (iv) the lack of an anxiety-specific negative cognition scale for concurrent validity analysis, (v) the comorbid psychiatric diagnoses of the participants, and (vi) the use of only self-report measures.

In conclusion, the current study has provided evidence that the Turkish version of the CCL is a reliable and valid instrument to assess NATs in a clinical outpatient sample.