Factor structure of the neurocognitive tests: an application of the confirmative factor analysis in stabilized schizophrenia patients.

The purpose of the present study was to identify the factor structure of neurocognitive tests used on schizophrenia patients by using the confirmative factor analysis, and to assess the factor score differences of schizophrenia patients and healthy controls. Comprehensive neurocognitive tests were administered to stabilized schizophrenia patients (N=114) and healthy controls (N=120). In the results of factor analyses on patients, the multifactorial-6-factor model, which included the speed of processing, working memory, verbal learning and memory, visual learning and memory, attention/vigilance, and reasoning/problem solving as suggested by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS), showed the better goodness of fit than any of the other models tested. And assessing the group differences of factor scores, we found the patients performed worse than the controls in all factors, but the result showed meaningful variations of impairments across the cognitive factors. Our study identifies the six major domains with multifactorial structure of cognitive abilities in schizophrenia patients and confirms the distinctive impairment patterns of each cognitive domain. These results may have utility in better understanding the pathology of schizophrenia as well as in genetic studies.

INTRODUCTION

The deterioration of cognitive functioning is a crucial feature of schizophrenia patients, and many of the meta-analyses have suggested a wide-range of cognitive impairments in schizophrenia patients (1).

However, there are inconsistent results regarding the cognitive architecture in schizophrenia. One of the reasons is that the human cognitive system is very complex. Although theoretical considerations, studies of brain localization, and developments in neuroscience are continuously made, the distinctive components of the cognitive system and their relations with one another are difficult to confirm. Second, it is not guarantee that neurocognitive tests assess only the specific cognitive domains or brain regions that they were designed to test. Since the performances of individual tests are influenced by a number of cognitive abilities, it is not uncommon that a test used in assessing a specific cognitive domain in one study is assigned to a different domain in another study (2). Lastly, the most important reason is that it is difficult to compare the results of previous studies because of the differences in their materials and methods. Specially, wide variations in the type of tests and the total number of tests administered constitute an important potential obstacle in making parallel comparisons, because the results of the factor analysis may have substantial differences due to the composition of variables (3). Recent factor analysis studies have yielded inconsistent factor solutions, in which the number of factors identified range from 2 to 14 (4).

Despite these limitations, identifying the cognitive domains is a crucial step in the research of schizophrenia. These domains have been applied in making a diagnosis, evaluating a course of illness and used as an endophenotype in genetic studies. Therefore, various neurocognitive batteries, including diverse domains, have been developed (5, 6).

One of the promising neurocognitive batteries is the MCCB (MATRICS Consensus Cognitive Battery). The National Institute of Mental Health (NIMH) of the United States developed the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative in order to provide a consensus of the crucial cognitive domains of schizophrenia (7). Based on the extensive empirical evidence, literature, and discussions by experts, the MATRICS Neurocognition Committee suggests seven cognitive factors of schizophrenia and the tests that are required in order to measure each of these domains (8).

However, another controversy exists regarding the structure of cognition in schizophrenia. Most factor analyses research treat cognitive factors in schizophrenia as 'separable' and 'discrete'. However, some studies emphasized that the general cognitive ability is defined as the 'g' factor or have suggested that some of the sub-domains do not need to be divided (2, 9). Dickinson et al. (9) has found that cognitive ability appears to be more unitary in schizophrenia than in healthy subjects. They suggested that the hierarchical model (has a higher order latent factor, representing general cognitive ability) is more appropriate than the multifactorial model (each of the latent factors cause performance on the individual tests) to test the cognitive structure of schizophrenia.

The goal of the present study was to examine the factor structure of neurocognitive tests in schizophrenia patient using confirmative factor analysis (CFA). First, we composed the neurocognitive battery including factors suggested by MATRICS and conducted to examine that MATRICS model fits to the empirical data. Second, we analyzed which of the multifactorial model or the hierarchical model is better appropriate for understanding schizophrenia cognitive structure. Additionally, we administered the same battery to healthy control subjects and assessed the factor scores differences between the groups to validate the distinctive impairments patterns and discuss the utility of factor scores.

MATERIALS AND METHODS

Participants

The study participants were comprised of 114 schizophrenia patients and 120 healthy controls. The patients were recruited from the outpatient clinics of the Samsung Medical Center and the Yong-In Mental Hospital. Diagnosis of patients was confirmed by the consensus among the researchers of this study using the Diagnostic and Statistical Manual of Mental Disorder-Fourth edition (DSM-IV) criteria based on observation, case notes, and the Korean Version of the Diagnostic Interview for Genetic Studies (DIGS-K) (10). Only clinical stabilized patients were included in this research in order to reduce confounding factors such as the severity of symptoms. All patients had a 1) current Clinical Global Impression (CGI) score below 3, 2) no psychotic symptom exacerbation, and 3) no changes in medication for at least three months prior to the time of the assessment. We excluded patients with a concurrent axis I diagnosis of DSM-IV, which is a current or past medical disease that is likely to have a significant effect on the central nervous system. Controls were free of all Axis I psychiatric disorders and familiar loading to second-degree relatives, and did not have a medical disorder that may alter cognitive functioning. Written informed consent was obtained from all of the subjects after a complete explanation of the study was provided. The study was approved by the institutional review boards of the Samsung Medical Center and the Yong-In Mental Hospital. Demographic and clinical characteristics of the participants are shown in Table 1. Differences in age, education, and the premorbid intellectual functioning, which was estimated from the Korean version of the Wechsler Adult Intelligence Scale (K-WAIS) vocabulary score, were considered in the group comparison analyses. This study was approved by the Institutional Review Board (2004-09-36).

Table 1

Demographic and clinical characteristics of the participants

K-WAIS, Korean version of the Wechsler Adult Intelligence Scale; CGI, Clinical Global Impression.

Neurocognitive assessment

The neurocognitive battery was composed of comprehensive tests that correspond to the six cognitive domains of the MATRICS, which were the speed of processing, working memory, verbal learning and memory, visual learning and memory, attention/vigilance, and reasoning/problem solving. Social cognition, which is the 7th domain of the MATRICS, was not included since the appropriate tools were not validated in Koreans. According to the psychometric characteristic criteria suggested by the MATRICS (8) and previous analysis of multicollinearity (11), a total of 19 variables were employed for the statistical analyses (Table 2).

Table 2

Tests grouped by cognitive domain

*using scaled scores in analyses.

K-WAIS, Korean version of the Wechsler Adult Intelligence Scale; K-AVLT, Korean version of Auditory Verbal Learning Test; K-CFT, Korean version of Complex Figure Test; DS-CPT, Degraded Stimulus-Continuous Performance Test.

Competing models

Fig. 1 shows the 6-factor model and the five nested models. The 1-factor model evaluated the general cognitive ability of all the tests (12). The 2-factor model included both the verbal and the visual ability based on Wechsler's original concepts (13). The 3-factor model was composed of the verbal ability, visual processing, and the speed of processing that is consistent with the factor analyses from recent versions of the Wechsler intelligence scales (14, 15). The 4-factor model does not have exactly the same prior theories, but is based on the studies that have separated the attention/working memory from the verbal ability in the intelligence tests of both adults and children (16, 17). The 5-factor model was predicted by the CFA study of Gladsjo et al. (18) with the exception of the 'verbal crystallized' factor. Finally, the 6-factor model is our hypothetical model that was recommended by MATRICS.

Fig. 1

6-factor models recommend by MATRICS and alternative models.

The 4-factor, 5-factor, and 6-factor models were also divided into two competing models that have either a multifactorial structure or a hierarchical structure (Fig. 2). The multifactorial model has separate latent cognitive factors that cause performance on the individual tests, and these latent factors can be intercorrelated. The hierarchical model has a single second order latent factor which causes performance in different cognitive domains (9).

Fig. 2

Representation of the hierarchical and multifactorial models.

Statistical analyses

The structure of the neurocognitive tests was determined via the CFA. Only data from the patients were included in the CFA. Prior to analyses, the data was evaluated for multivariate nonnormality (19). The multivariate kurtosis was significantly high (12.918), and because of this the Maximum likelihood Estimation (MLE) extraction with Bollen-Stine bootstrapping (20) was used in order to meet the multivariate normality assumption of MLE. Error variables that shared a common method effect were allowed to correlate to improve the fitting of the model (18).

According to the CFA-derived best fit model structure, the factor scores of the patients and the healthy controls were compared using the MANCOVA analysis while controlling for age, education and premorbid intellectual functioning. Factor scores were determined by averaging the standardized Z scores by using the means and standard deviations of the healthy control group (21, 22). The analyses in this study were conducted using the SPSS 15.0 and the AMOS 7.0 software.

RESULTS

Confirmatory factor analyses of the schizophrenia patients

We examined the goodness of fit and the parameters by adding four pairs of correlated errors (Tail-Making-Test A and Tail-Making-Test B, letter fluency and category fluency, 1-back test and 2-back test, and 3 letter Span and 12 letter Span) based on prior studies and the modification index used to control for the method effect. The fit indices for the tested models are shown in Table 3. The multifactorial-6-factor model had the best fit compared to any of the other competing models. The χ2/df and the Root Mean Square Error of Approximation (RMSEA) for the multifactorial-6-factor model met the established guideline (23) for adequate fit (χ2/df=1.57, RMSEA=0.071). Other fit statistics of all models failed to meet the criteria standards, but the multifactorial-6-factor model was relatively appropriate (CFI=0.889, GFI=0.843, NFI=0.755, TLI=0.857).

Table 3

Goodness-of-fit indices of overall models

An acceptable model fit was defined by the following criteria: χ2/df (1-2), GFI (≥0.90), NFI (≥0.90 ), CFI (≥0.90), TLI (≥0.90), RMSEA (≥0.08). CFI, comparative fit index; GFI, goodness of fit index; NFI, normed fit index; TLI, Tucker-Lewis index; RMSEA, root mean square error of approximation.

Table 4 shows the factor loadings of the multifactorial-6-factor model. The factor loading of K-WAIS digit span-backward on 'Working memory' is relative low, but the loadings of all the observed variances of the multifactorial-6-factor model are significant (standardized regression weight=0.274-0.986).

Table 4

Factor loading for the multifactorial-6-factor model

SP, speed of processing; WM, working memory; VM, verbal learning&memory; VSM, visual learning&memory; AV, attention/vigilance; RP, reasoning/problem solving; K-WAIS, Korean version of the Wechsler Adult Intelligence Scale; K-AVLT, Korean version of Auditory Verbal Learning Test; K-CFT, Korean version of Complex Figure Test; DS-CPT, Degraded Stimulus-Continuous Performance Test.

Differences of factor scores between schizophrenia patients and control subjects

In Table 5, the mean z-scores and the standard deviations for the cognitive domains were compared. All of the factor scores were significantly lower in the patient group (F [6,211]=23.698, P<0.001, Wilk's Lamda=0.597; partial eta square=0.403). More pronouncing deficits were observed in 'Processing of information' (mean Z score=-1.96) and 'Verbal learning and memory' (mean Z score=-1.28). Patients showed moderate impairments in 'Working memory' (mean Z score=-1.17) and 'Attention/Vigilance' (mean Z score=-1.15). 'Visual learning and memory' (mean Z score=-0.69) and 'Reasoning and problem solving' (mean Z score=-0.79) were relatively mildly impaired.

Table 5

Comparison of z-scores on cognitive domains between patients and healthy controls

Age, education, premorbid intellectuan functioning as covariate.

P<0.008 to set the critical alpha level after Bonferroni correction divied by the number of domains (0.05/6).

DISCUSSION

The purpose of the present study was to assess the factor structure of neurocognitive tests used on schizophrenia patients by using the confirmatory factor analysis. And we compared the differences of the factor scores between schizophrenic patients and control subjects.

First, our results confirmed that the 6-factor model that was proposed by MATRICS is more appropriate than any of the competing models that were tested. Although this result is difficult to compare to the results of previous factor analytic studies directly, all six domains of this model have been replicated as major cognitive functions of schizophrenia. 'Speed of processing' was composed of tests that were shown to distinguish impairments in schizophrenia patients and their family. The construction validity and neurological mechanism of this is difficult to explain, but all of the tests that involve both perceptual and motor components and the many factor analysis studies support that this is one of the most basic domains (24). 'Working memory' was defined as the capacity to simultaneously store, process and manipulate information, and deficits of this in schizophrenia patients are consistently demonstrated across a diverse range of tasks. In our results, the factor loading of K-WAIS digit span-backward on this dimension was low. It is a reason that this task also thought of as a measure of 'Attention/Vigilance' in other studies, that is, the validity of this task on 'Working memory' is relative lower, so less loads on this dimension. 'Reasoning and problem solving' are the skills in planning and decision making, and meaningfully relate to the real world functioning of schizophrenia patients. 'Reasoning and problem solving' and 'Working memory' can be referred together as 'executive functioning'. However, using 'executive functioning' as a unitary cognitive operation has been criticized, because it may require a higher ability level to connect and control the information between subsystems (2). This is why MATRICS tried to divide 'executive functioning' into two domains and our study supports this idea. An abundant number of studies have revealed both stable and wide memory deficits in schizophrenia patients. Memory functioning can be divided in to a lot of sub domains, most especially as 'Verbal memory' and 'Visual memory', which has been conceptualized as separable areas and replicated to have a respective course of impairments. The 'Attention/Vigilance' is described in the early clinical accounts of schizophrenia, and is one of the most frequently studied cognitive deficits and show relationships to everyday functioning (3).

Second, we found that the multifactorial model provided a better fit than the hierarchical model. Our research confirmed the assumption that cognitive dimensions are 'independent'. Recent studies that used novel data-mining methods with no prior assumptions also supported 'separable' factors (25). The multifactorial structure was verified by the distinct relationships between the factors and the clinical or functional capacity variables. Longitudinal studies have indentified that the response to the course and type of anti-psychotic treatment for each type of cognitive domain are diverse (26).

Additionally, we can also validate the multifactorial model by identifying the differences in impairments between the cognitive factors. Schizophrenia patients performed worse than the controls in all of the domains tested, but the result showed meaningful variations of deficits. Similar to a number of previous studies, our results showed that the most severe impairments were found in the 'Speed of processing' and the 'Verbal memory'. These impairments are either stable or progressively deteriorating (27, 28). Deficits of the 'Working memory' and the 'Attention/Vigilance' were moderate since the early stages of the illness, and the 'Visual memory' is relatively preserved (29). In our results, the impairments of the 'Reasoning/problem solving' are relatively mild, and this finding is inconsistent with the results of some previous studies. It may be a reason that some previous studies put 'reasoning/problem solving' and 'executive functioning' in the same category, so used more complex level tests diverse and high order cognitive abilities needed.

Reducing the variables in a large cognitive battery to major cognitive abilities may reflect latent traits and may result in having better psychometric properties (4). Clarifying the features of cognition by the progress of illness may have utility in reducing the type 1 or 2 error in making a diagnosis and would result in a better understanding of the pathology of schizophrenia. This also has more implications on genetic studies since specific cognitive deficits can be associated with specific genes, brain function or pharmacological mechanism more definitively.

Our study has several limitations. First, the multifactorial-6-factor model did not fulfill all of the goodness-of-fit criteria. This may have been attributed to an insufficient sample size to case per variable ratio for the factor analysis. Another reason may be due to the differences in the tests and grouping measures that were used in our batteries than what was recommended by the MATRICS. This limitation is not uncommon in factor analysis when using a large number of variables that are derived from the multiplicity tests. Second, the social cognition was not included even though it was mentioned as the most recent domain of interest by the MATRICS. The validation and development of tests to measure social cognition in Korean needs to be completed before it can be studied in a Korean population. Actually, other new cognitive domains and their interactions need to also be studied. Recently, the CNTRICS (Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia) project was initiated to identify a set of cognitive systems and to develop optimized measurements based on the cognitive neuroscience approach (30). It is important to recognize that the results of our study are tentative conclusions of the cognitive architecture of schizophrenia in order to design and interpret cognitive test batteries. Defining new cognitive domains, elaborating tests, and identifying the multi-level and dynamic interactions of all known domains must be further researched.

In summary, using the confirmatory factor analysis of neurocognitive tests on schizophrenia patients, our study identifies that the multifactorial-6-model suggested by MATRICS based on the prior studies and theoretical considerations is best fit the data. Also we confirm the distinctive deficits patterns across the cognitive domains by assessing the differences of factor scores between schizophrenia patients and healthy controls.