A meta-analysis of cognitive impairment following adult cancer chemotherapy.

OBJECTIVE: Chemotherapy-induced cognitive impairments are reported by many cancer survivors. Research to date has not provided a clear description of their nature, extent, mechanisms, and duration. To investigate the impairments and factors that could influence their identification and severity, the present meta-analysis brings together research on this topic in adult cancer patients.
METHOD: Our random-model meta-analysis includes 44 studies investigating the cognitive performance of adults treated with chemotherapy for non-central nervous system malignancies, primarily breast and testicular cancer. We conducted several subgroup analyses to identify the level of cognitive impairments in longitudinal and cross-sectional studies. We also pursued several multilevel model regressions to investigate the impact of methodological (study quality) and clinical moderators (diagnosis, age, time since treatment) on the observed effect sizes.
RESULTS: Cognitive impairments were found in cross-sectional studies in immediate free recall, delayed memory, verbal memory, delayed recognition memory, selective attention, and attention capacity. Surprisingly, prior to chemotherapy, patients performed better than matched controls. In longitudinal studies, patients' performance increased from baseline to follow-up, an effect that was stronger in patients than controls. None of the chosen moderators influenced the magnitude of estimated summary effect sizes.
CONCLUSIONS: The likelihood to identify impairments rests on the type of design employed, as memory and attention impairments are only detected in cross-sectional studies. We discuss the lack of significant impact of moderators on the effect sizes despite the heterogeneity of results, while providing recommendations toward decreasing the heterogeneity in future studies. (c) 2014 APA, all rights reserved.

Cognitive impairments may contribute to a lower quality of life following cancer diagnosis and treatment (Short, Vasey, & Tunceli, 2005). Despite this, health care systems in many countries do not have the appropriate resources to help people cope with chemotherapy-induced cognitive impairments (Ferguson et al., 2007). Furthermore, the continuous care needs of survivors, in terms of their cognitive deficits, and how these might relate to potentially higher distress levels and lower quality of life are yet to be identified. This is a problem given the increasing number of people living with and beyond cancer (Maddams et al., 2009). The limited knowledge regarding this phenomenon may be a result of several factors.

First, chemotherapy-induced cognitive changes in adult patients do not have a long research history. Consequently, guidelines for conducting neuropsychological research with former adult patients were proposed only recently by the International Cancer and Cognition Taskforce (ICCTF) (Vardy, Wefel, Ahles, Tannock, & Schagen, 2008). Second, research in this area may have possibly been hampered by inconsistencies in previous findings regarding cognitive impairments. On the one hand, there are differences between the degrees of objective impairment reported by different studies, being found in 12% to 68% of cancer survivors (Ahles & Saykin, 2007; Shilling, Jenkins, & Trapala, 2006). On the other hand, subjective impairments are reported by up to 80% of these patients (Kohli et al., 2006). Inconsistencies in the percentage and types of impairments reported by the literature yielded some uncertainty about which functions are impaired, and may have reduced the emphasis on evidence-based intervention strategies to help patients overcome these problems (Ferguson et al., 2007).

Previous reviews have claimed that the lack of cohesion within the literature might stem from variability in several factors, including participant demographics such as age and gender (Ahles et al., 2003), treatment protocols (Freeman & Broshek, 2002; Hurria, Somlo, & Ahles, 2007; Jansen, Miaskowski, Dodd, Dowling, & Kramer, 2005), and variability in the neuropsychological tests used in assessments (Hutchinson, Hosking, Kichenadasse, Mattiske, & Wilson, 2012; Jansen, Miaskowski, Dodd, & Dowling, 2007). Additional confounding may be due to limited consideration of practice effects (Vardy et al., 2008), whether patients are compared to test norms or matched control participants, and whether the matching is solely on age and gender or also on intelligence and education, (Anderson-Hanley, Sherman, Riggs, Agocha, & Compas, 2003; Jansen et al., 2005), and differences in the statistical cut-offs for defining the impairments (Hurria et al., 2007; Wefel, Vardy, Ahles, & Schagen, 2011). However, other than the impact of comparing patient results to norms or controls, most of these potential moderators have not been systematically investigated in relation to the degree of observed impairments. We will further give a brief account of the results obtained by previous literature that sought to identify the nature and extent of impairments.

Conclusions From Previous Meta-Analyses

Four previous meta-analyses have summarized the cognitive outcomes of earlier studies (Anderson-Hanley et al., 2003; Falleti, Sanfilippo, Maruff, Weih, & Phillips, 2005; Jansen et al., 2005; Stewart, Bielajew, Collins, Parkinson, & Tomiak, 2006). Relative to controls, survivors exhibited a broad range of mild to moderate cognitive deficits in attention, information processing, verbal and visual, long-term and working memory, spatial skills, language, executive and motor functioning, summarized in Figure 1. While the four analyses agreed on the direction of the effects, there was less agreement on their magnitude, despite analyzing approximately the same literature; for example, the effect size of speed of processing impairment ranges from low (Stewart et al., 2006), through medium (Jansen et al., 2005), to large (Anderson-Hanley et al., 2003).

Figure 1

Forest plot of summary effect sizes obtained by four previous meta-analyses of studies with adults. Note. We report the type of cognitive function reported by each study, as well as the reported effect size, and its 95% CI. Falleti et al. (2005) did not report confidence intervals.

In addition to variability in designs and reported outcomes in primary studies, there was also variation between the meta-analyses in the reporting of key methodological factors. As an example, these meta-analyses reported 11, 12, 18, and 20 items from the 27-item checklist of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Kyoko, Yoshitoku, & Toyonori, 2011). The items missed most frequently were those pertaining to the process of study screening, the reporting of confidence intervals, consistency, and risk of bias analyses (Anderson-Hanley et al., 2003; Falleti et al., 2005; Jansen et al., 2005; Stewart et al., 2006). Detailed literature search techniques were not always reported and the meta-analyses were based on small numbers of studies (n = 6, 7, 16, 29, respectively). When subgroup analyses were reported, they included even fewer studies (Stewart et al., 2006), although it is not generally recommended to run summary effect size analyses with a very small number of studies (Borenstein, Hedges, Higgins, & Rothstein, 2009).

There was also significant variability in the way neuropsychological test scores were pooled into cognitive functions and reported as single, general scores (i.e., verbal, short-term memory (STM), attention). Yet, current understanding of memory and attention is far more detailed than this (Strauss, Sherman, & Spreen, 2006). Such simplification of results, through the combination of very different measures, might impact the observed effect sizes, and the heterogeneity of results.

The Present Meta-Analysis

The present meta-analysis is a synthesis of current literature (K = 44) reporting cognitive functioning in adult cancer survivors treated with chemotherapy (described in online supplementary materials). We examined the extent of cognitive impairment but, compared to previous meta-analyses, we also explored potential sources of methodological and clinical heterogeneity, which might have influenced the results obtained within the literature.

We pursued two types of subgroup analyses. First, we grouped test scores into constructs based on well-established guidelines (Strauss et al., 2006) with a clear distinction between different memory and attention types. Table 1 summarizes the tests grouped within each neuropsychological function, as well as the number of studies (K) and effect sizes (N) included in the analyses.

Table 1

List of Cognitive Tests Included in Each Function

Aggregate constructs	Specific constructs	Tests used	K	N
Note. The number of studies (K) and effect size estimates (N) within all the analyses, before the multiple outcomes transformations. (MMSE = Mini Mental State Examination; WAIS = Wechsler Adult Intelligence Scale; WMS = Wechsler Memory Scale; WTAR = Wechsler Test of Adult Reading; CVLT = California Verbal Learning Test; RAVLT = Rey Auditory Verbal Learning test; HVLT = Hopkins Verbal Learning Test; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; VLMT = Verbal Learning and Memory Test; VSRT = Verbal Selective Reminding Test; NVSRT = Non-Verbal Selective Reminding Test; ROCFT = Rey-Osterrieth Complex Figure Test; TEA = Test of Everyday Attention; PASAT = Paced Auditory Serial Addition Test; WCST = Wisconsin Card Sorting Test; COWA = Controlled Oral Word Associations).
	Full-scale IQ	Groeninger Intelligence Scale, MMSE, WAIS.	5	5
Memory		All memory tests regardless of retention interval (immediate or delayed), test format (free recall or recognition), and modality (visual or verbal).	37	189
Verbal memory		Verbal memory tests regardless of retention interval and test format (free recall or recognition).	31	122
Visual memory		Visual memory tests regardless of retention interval and test format (free recall or recognition).	20	57
Immediate free recall		Immediate memory tests regardless of modality.	26	47
Delayed memory		Delayed memory tests, regardless of modality and test format.	26	69
Delayed recognition		All recognition memory tests regardless of modality.	12	19
	Verbal immediate free recall	Logical memory I, CVLT/RAVLT/HVLT/Rey 15, WMS Verbal memory immediate, RBANS Immediate memory, VLMT 1–5, VSRT Short term, Encoding/recall correct.	24	48
	Verbal delayed free recall	Logical memory II, CVLT/RAVLT/15 Rey Delayed, RBANS Delayed memory, WMS Delayed recall, VSRT Delayed.	24	48
	Verbal delayed recognition	RAVTL/CVLT/Rey 15 recognition, HVLT Discrimination, Paired associates recognition.	9	21
	Visual immediate free recall	Logical memory I, variants of CVLT/RAVLT/HVLT/VLMT, RBANS Immediate memory, Encoding/Recall correct, WMS Visual memory immediate.	17	27
	Visual delayed free recall	Visual reproduction II, Family pictures II, ROCFT delayed, NVSRT delayed, WMS Visual memory delayed.	15	22
	Visual delayed recognition	Visual reproduction recognition, ROCFT recognition, Visual association test.	8	7
Attention		Includes all attention tests.	11	17
	Focused attention	Trails A, Stroop, Digit symbol, Symbol search, Symbol modalities, D2, Continuous performance tests, Visual search tests.	28	74
	Selective attention	D2, Fepsy binary, Go/No go selective attention, TEA Auditory/visual elevator, Ruff 2&7.	10	24
	Attention capacity	Letter-number cancellation/sequencing, PASAT, Digit span, Visual span Forward, Sentence repetition.	19	59
	Executive functions	Stroop, Trail B, WCST, Tower of London, Consonant Trigrams, COWA or variants.	33	147
	Verbal abilities	Lexical/Semantic search, Boston naming test, WAIS/WRAT Reading, RBANS Language.	8	15
	Spatial abilities	Block design, ROCFT-Copy, RBANS Visuospatial (Figure copy and Line orientation).	15	18
	Arithmetic	WAIS, WISC, WPPSI and any other mathematical achievement tests.	10	17
Motor functions		Pegboard, Fingertapping, Grip strength dominant and non-dominant.	16	45

Second, we divided studies into subgroups based on their designs, as cross-sectional studies and longitudinal studies might have different sources of bias. For example, in cross-sectional studies, the effective matching between patients and controls is crucial to the identification of real impairments, while longitudinal designs are particularly vulnerable to practice effects, both when tests have alternative formats and especially when they do not.

We further investigated the influence of two specific sources of bias, or moderators (Rosenthal & DiMatteo, 2001), which have not been considered by previous literature and could have influenced the observed effect sizes. The first is a methodological moderator, the quality of studies. The methods of conducting and reporting the results of primary studies, the quality of participant matching, the type of cognitive tests used, as well as availability of tests with alternative forms, are integral parts of their quality having a potential influence on the results of subsequent meta-analyses. Thus, we ran quality assessments of the studies and included the scores as potential methodological sources of heterogeneity.

The second sources of bias are clinical, related to the participant characteristics reported by primary studies and suggested by previous literature to have an impact on cognitive test results. These are the type of diagnosis, age of participants, and time since treatment. There could be several additional factors that might have had a significant impact, but those were either not reported (i.e., test results on treatment types or genders), reported inconsistently (i.e., types of treatments, time since diagnosis) or reported through different test scores (i.e., premorbid intelligence level of matched groups). The diagnosis was chosen as a proxy for the types of treatment and genders of the participants. The age of participants was chosen due to evidence from pediatric cancer studies that a younger age may be a vulnerability factor for cognitive impairments (von der Weid et al., 2003). Finally, some behavioural studies fail to show differences in functioning in breast cancer patients tested at several time points after treatment (Hermelink et al., 2007). Given that some impairments may fade with time, the time lapsed from treatment to assessment might be another significant factor influencing cognitive test scores.

Our meta-analysis followed the robust and comprehensive guidelines of the Cochrane Collaboration (Higgins & Green, 2008) for conducting systematic reviews and meta-analyses. Additionally, due to the problems with methods such as the fail-safe N, which assumes that the effect sizes of missing studies would be zero (Borenstein et al., 2009), we performed a regression-based publication bias analysis using Egger’s method to account for potentially unreported data (Egger, Davey Smith, Schneider, & Minder, 1997; Sterne, Gavaghan, & Egger, 2000).

Methodology

Search Strategies

The relevant literature was examined by one person (OCL) through a search of the electronic databases (PubMed, Ebsco, Web of Science, PsychInfo, PRISMA, Cochrane) using the following search terms: (cancer OR chemotherapy) AND (cognition OR neuropsychology) AND (adults). We also conducted the search by replacing the words (cancer OR chemotherapy) with the names of chemotherapy drugs (i.e., doxorubicin, cyclophosphamide, etc.) and by replacing the words (cognition OR neuropsychology) with names of specific cognitive functions (i.e., attention, memory, verbal memory, executive functions, etc.). The reference lists of reviews were visually scanned and key journals of the International Psycho-Oncology Society, and conference proceedings were hand searched for additional articles not detected by the literature search (list included in online supplementary materials).

Inclusion and Exclusion Criteria

The study eligibility criteria are described in Table 2. They were driven by the participant/intervention/comparison/outcomes/study design elements (Higgins & Green, 2008), while accommodating the ICCTF guidelines for studies in the field (Vardy et al., 2008).

Table 2

Criteria for Including Studies in the Meta-analysis

	INCLUSION	EXCLUSION
Population	Patients exposed to chemotherapy.	Patients with central nervous system tumors.
Intervention	Patients exposed to chemotherapy.	Studies that test the effect of drugs other than chemotherapy (i.e. hormonal treatments).
		Patients exposed to CNS-directed radiotherapy.
Comparison/control group	Studies comparing patients with norms, healthy controls, or cancer patients who were not treated with chemotherapy.	Studies that did not report results of any control group.
Outcomes	Articles reporting means and standard deviations of at least one neuropsychological test. See Table 1 for a description of the type of tests.	Duplicate results (i.e. articles based on dissertations) and studies only reporting changes in psychosocial functioning such as quality of life. Studies not reporting means and standard deviations on the tests.
Study design	Longitudinal and cross-sectional studies.	Case-studies were excluded because the design is rarely used to examine intended effects of a treatment.

Our search included all studies from 1980 to January 2011. We did not include unpublished data, or articles written in languages other than English. When studies did not report means and standard deviations, they were requested from the authors. If the data were not provided, we did not include the articles because of the nature of the software used for the initial analyses. Figure 2 depicts the search process that led to the inclusion of 44 studies.

Figure 2

Study selection flowchart.

Coding Procedures

Means and standard deviations for all cognitive tests were recorded for each individual study, alongside the study quality and participant level moderators (cancer type, mean age, and mean number of years since treatment). The scores reported for each cognitive test were extracted from each article (i.e., Rey-Osterrieth Complex Figure Test-Copy as a measure for visuospatial ability). These scores were then grouped within functions based on the guidelines suggested by Strauss et al. (2006).

The strengths and weaknesses of the studies included in the analyses were graded according to a quality assessment tool recommended by the Cochrane Collaboration. The Downs and Black scale (Downs & Black, 1998) contains 27 questions pertaining to both randomized and nonrandomized studies and looks at all aspects of data reporting and analysis. Three questions, referring to the blinding of participants and experimenters, were removed due to the lack of suitability in the context of this research. Six additional questions were added to accommodate the ICCTF guidelines: whether any patients were exposed to local or cranial radiotherapy, had CNS malignancies, if a control group was present, and the inclusion of ICCTF recommended neuropsychological and self-assessment tools. Thus, each study received a score between 0 and 31.

The quality assessment was performed blindly by one master coder (OCL) and an independent researcher (DF). Disagreements about the scores were resolved by consensus. Following consensus, an interclass correlation was performed because the scores were measured on an interval scale; it yielded an interrater correlation of 0.97 (p < .001), with scores ranging from 11 to 30.

Subgroup Analyses

Most studies reported multiple outcomes and multiple time-points scores for each cognitive function. This facilitated the separation of the data into subgroups based on the types of designs employed by individual studies. The resulting subgroup analyses were:

Postchemotherapy cross-sectional studies: patients versus controls, after treatment.

Patient longitudinal studies: patients at follow-up versus baseline.

Baseline cross-sectional studies: patients versus controls, before treatment.

Control longitudinal studies: control participants at follow-up versus baseline.

The results of the first two analyses triggered the analyses pertaining to the performance of patients versus controls before chemotherapy and the performance of controls in longitudinal assessments. Their aim was to determine whether patients were impaired before chemotherapy and whether practice effects were present in both patients and controls evaluated multiple times. Each analysis was based on at least four individual study estimates, which is sufficient to perform a meta-analysis (Patel, 1989).

Effect Size Estimations

We used the random effects model for each level of the analyses, in order to account for the high variability of the data (Overton, 1998). We have computed Hedge’s g standardized mean difference between groups as it provides a tighter estimate of the true effect size (Borenstein et al., 2009). Each study reported several scores for each cognitive test, for each comparison group, and for different time-points. Initially, we ran individual random model meta-analyses for all the data, and then in the four additional subgroups for each of the 22 cognitive functions. These initial analyses resulted in specific standard errors, and weights assigned to each outcome within each study. All of these calculations were run using Meta-Analyst (Wallace, Schmid, Lau, & Trikalinos, 2009).

In order to account for the dependency of data due to multiple outcomes and time points, we further calculated study-level composite effect sizes and variances for each set of outcomes reported by each study, per cognitive function. These were computed based on the formulas suggested by Borenstein et al. (2009). Finally, each study appeared once in the final summary effect size analyses. Studies reporting the same outcomes on different patient groups (i.e., lymphoma and breast cancer, Ahles et al., 2002, 2003), were analyzed as two separate studies because of the importance of the differences between diagnoses.

Effect Size Integration

In order to create the random effects model summary effect sizes we used an adapted version of the meta-analysis macros developed by Field and Gillett (2010). This enabled us to use the composite effect sizes, and variances previously computed. Moreover, it helped us correct for unequal sample sizes by using the minimum weight received by each study in the initial analyses, and to compute the I heterogeneity value.

The subgroup summary effect sizes are not directly comparable, thus they were interpreted as low if they were 0.2 or below, moderate between 0.2 and 0.5, and high above 0.8 (Cohen, 1992). However, as this is a general rule of thumb, effect sizes were also interpreted in the light of the specific literature on the topic of chemotherapy-induced cognitive impairments (Lipsey & Wilson, 2001), in which the effects are usually considered mild. For each analysis, we report the summary effect size, confidence intervals (95% CI), I, and the overall significance level representing the null hypothesis that the treatment effect is zero (Borenstein et al., 2009). The negative or positive valence of effect sizes denotes the direction of the effect of chemotherapy on cognitive function: negative if suggestive of impairments, and positive if suggestive of performance increases in one group relative to the other.

Publication and Selection Bias

The publication bias for each main cognitive function was assessed through Egger’s regression test (Egger et al., 1997). It estimates the asymmetry of the funnel plot due to underreporting of data through a linear regression comprised of a normalized effect size (divided by its standard error) and precision (inverse of standard error) We present the significance level of the intercept, which was considered to suggest publication bias if significant for p < .10.

Multilevel Moderator Analyses

Presently, there are two options available for conducting moderator analyses within a meta-analysis. The first and most highly used method was to run metaregressions for each function, each design, and each moderator separately with the restricted maximum likelihood macros developed by Lipsey and Wilson (2001). The advantage of this method was that we could run specific analyses for each subgroup. However, this method can lead to a decrease in variance and a high likelihood of an increased Type I error when comparing multivariate effect sizes. Due to this reason and the presence of a categorical moderator (diagnosis) which requires dummy coding, we also pursued a multilevel model analysis for all effect sizes and moderators (Hox, 2010). We will briefly report the results of the classical metaregressions, while focusing more on the results of the multilevel model approach.

The multilevel analyses were conducted with MLwin 2.1 (Rabash, Charlton, Browne, Healy, & Cameron, 2009) with the restricted maximum likelihood procedure. We used a 3-level model with the study outcomes (summary effect sizes) as the first level, cognitive functions as the second level, and the studies as the third level. The moderators were the ones described above: study quality, diagnosis (coded as a dummy variable), age of participants, and time since treatment.

We will first report the intercept-only model, when no predictors are included. This is described by the equation:

ES refers to the effect size for outcome i from study j, β is the value of the intercept (average effect size for an average outcome), u is the random error at level 2, and e is the random error residual at level 1. The variance of u suggests the variability in effect sizes.

In the moderator analyses the equations take the form of:

All the parameters represent the same values as in the empty intercept model, while the β value represents the slope of the regression, suggesting the strength and direction of the change in effect size for a one-unit change of the moderator. All the analyses were run with predictors centered on their grand mean, to reduce the possibility of correlations between the intercept and predictors, as well as between levels (Kreft & de Leeuw, 1998).

Results

Study Characteristics

We analyzed data pertaining to the neuropsychological evaluations of 1,940 adult patients and 2,000 controls. In total, 30 out of 44 studies (70%) included only breast cancer patients. The remainder included patients with testicular cancer, or lymphoma, or other hematological malignancies. Mean participant age was 51.57 (SD = 6.29), and 75% of the studies included only female participants. All studies evaluated patients at an average of 2 years posttreatment (SD = 2.52). In online supplementary materials we described the characteristics of each study included in our meta-analysis and the associated moderators.

Analysis of Effect Sizes Across All Subgroups

First, we compared the performance of all patient participants after chemotherapy to the performance of controls. Analyses were undertaken for all patients, irrespective of the comparison group—healthy controls or their own baseline performance—to allow comparison with previous meta-analyses. Effect sizes were small, had broad confidence intervals, and high heterogeneity (see Figure 3). Patients had statistically significant performance increases (positive summary effect sizes) for visual memory and visual immediate free recall. Patients’ performance was significantly reduced (negative summary effect sizes) only for selective attention.

Figure 3

Forest plot of summary effect sizes in adult patients compared to any control group. Note. We report the Hedge’s g effect size, the 95% CI, I, p as the significance level of the analysis, k as the number of studies in the analysis, and the significance level of the intercept in Egger’s test. * p < .05.

Analyses by Study Design

Cross-sectional designs

We performed separate analyses for cross-sectional studies, at posttreatment and baseline. Following chemotherapy, patients exhibited significant low to moderate impairments relative to controls (Figures 4 and 5). These were observed in memory, immediate free recall, delayed memory, delayed recognition, verbal memory, verbal immediate free recall, verbal delayed free recall, verbal delayed recognition, selective attention, and capacity of attention. Summary effect sizes of other cognitive functions were not statistically significant.

Figure 4

Forest plot of longitudinal and cross-sectional study effect sizes in adult patients for memory functioning only. Note. k = number studies. Full diamonds = cross-sectional effect sizes; empty diamonds = longitudinal effect sizes. * p < .05, ** p < .01, *** p < .001.

Figure 5

Forest plot of longitudinal and cross-sectional study effect sizes in adult patients for cognitive functions other than memory. Note. k = number studies. Full diamonds = cross-sectional effect sizes; empty diamonds = longitudinal effect sizes. * p < .05, ** p < .01, *** p < .001.

At baseline, before chemotherapy, patients performed better than controls, across most cognitive functions (see Figure 6). Effect sizes were moderate to high, and statistically significant, but had high heterogeneity values. Superior patient performance was observed in memory, attention, executive functions, spatial abilities, and verbal abilities. Due to an absence of reported outcome data we could not compute cognitive function effect sizes for 12 functions. Results were not significant for the remaining four functions, two of which (verbal memory and attention capacity) had negative values, suggestive of potential impairments.

Figure 6

Forest plot effect sizes of patients versus controls at baseline for all cognitive functions. Note. We report the Hedge’s g effect size, the 95% CI, I, p as the significance level of the analysis, k as the number of studies in the analysis, and the significance level of the intercept in Egger’s test. ** p < .01, *** p < .001.

To summarize the findings from cross-sectional designs, as expected, patients performed worse than controls after treatment. Contrary to expectations, patients performed better than controls at baseline, before treatment began. This pattern was observed across most cognitive functions, even in cases where it was not statistically significant. Figure 7 visually depicts the difference between our findings and what we predicted on the basis of available literature.

Figure 7

Pattern of significant effect sizes results for memory in cross-sectional studies. Note. Straight line represents the relationship between effect sizes obtained by patients versus control at pretreatment and posttreatment.

Longitudinal designs

When longitudinal studies were analyzed separately, there was a clear improvement in patients following chemotherapy, compared to baseline. This finding was true across most cognitive functions we were able to analyze (Figures 4 and 5). The improvements in performance were statistically significant for immediate free recall, verbal immediate free recall, visual immediate free recall, visual delayed memory, focused attention, capacity of attention, and verbal abilities. Effect sizes of other cognitive functions did not reach statistical significance. Notably, heterogeneity was higher in studies measuring longitudinal changes than in cross-sectional studies.

Longitudinal data from healthy control participants were infrequently reported (see Figure 8). As a result, effect sizes could only be computed for five of the 22 cognitive functions, each analysis including up to a maximum of seven studies. The analyses were restricted to those reporting patient and control comparison at different time points. Controls performed significantly better at follow-up than at baseline on memory and visual memory. These analyses were less heterogeneous, despite the smaller number of studies reporting longitudinal outcomes for control participants.

Figure 8

Forest plot effect sizes of controls at follow-up versus baseline for all cognitive functions. Note. We report the Hedge’s g effect size, the 95% CI, I, p as the significance level of the analysis, k as the number of studies in the analysis, and the significance level of the intercept in Egger’s test. * p < .05, ** p < .01.

To summarize, contrary to expectations, patients performed better after chemotherapy than before. The only exceptions were spatial abilities and selective attention, which had negative values, but were not statistically significant. Controls also improved, but the effect sizes were not as large as the ones estimated in patients. Because the two sets of effect sizes within the two subgroups are not directly comparable, we depict the expected versus observed values for controls and patients in longitudinal studies (see Figure 9). For all other cognitive functions, which were not significant, patients’ effect sizes were only negligibly reduced compared to those of controls (e.g., executive functions, patients g = .02 and control g = .08).

Figure 9

Pattern of significant effect sizes for visual memory in longitudinal studies Note. Straight line represents the relationship between effect sizes obtained by patients at pretreatment versus posttreatment, and separately for healthy controls.

Moderator Analyses

The classical moderator analysis, which was run on Lipsey and Wilson’s (2001) macros, had the advantage of analyzing the impact of each moderator on each function within the two main designs. However, it had the disadvantage of comparing multiple effect sizes characterized by group dependencies, thus the variances of the results may have been underestimated and the significance values overestimated. Still, in R2 values, the quality of the studies, age, and time since treatment significantly explained between 38% and 69% of the variance of effect sizes in cross-sectional studies, and between 16% and 66% of the one in longitudinal studies. The most noteworthy results were those of the quality of studies influencing up to 34% of motor function effect size variance in cross-sectional studies, and time since treatment explaining 56% of the visual immediate free recall variance in longitudinal studies (data not presented in paper, available upon request).

Our multilevel model did not have the advantage of analyzing the results in subgroups, but for all available effect sizes. We will first report the result of the intercept-only model (see Tables 3 and 4 for all the coefficients). The first level was described by the summary effect sizes, the second level was represented by the cognitive functions, while the third level was the study itself. The intercept only model, independent from variances at the third level was estimated at .18 (standard error = .07). Compared to the z-critical value for p < .05, the residual variance was significant, 3.10 (.17). In other terms, the overall mean effect size, irrespective of the type of design, functions, or variances at the third level, was a low positive summary effect size, but there is a great amount of unexplained variance.

Table 3

Results of Multilevel Regression Analyses for Each Continuous Moderator

Moderator	β0 (SE)	β1(SE)	σu2 (SE)	σe2 (SE)	−2*loglikelihood
Note. We report K as the number of studies in the analysis, N as the number of effect sizes included, β0 as the value of the intercept, β1 as the value of the slope associated with a certain moderator, the associated standard errors, σu2 as the variance associated with the study level, σe2 the variance of the random error, and the −2*loglikelihood values.
Empty model (independent from study)	.18 (.07)	NA	NA	3.106 (.17)	2377.67
Empty model (dependent on study)	.13 (.11)	NA	.24 (.103)	2.793 (.16)	2344.65
Quality	.16 (.11)	.05 (.03)	.22 (.09)	2.791 (.16)	2341.55
Age	.13 (.11)	.01 (.01)	.24 (.10)	2.793 (.16)	2343.56
Time	.14 (.11)	−.04 (.04)	.23 (.10)	2.796 (.16)	2343.69
					K = 44	N = 599

Table 4

Results of Multilevel Regression Analyses for Each Type of Cancer

β0 (SE)	Breast cancer	Testicular cancer	Mixed diagnoses
	(β1/SE)	(β1/SE)	(β1/SE)
Note. We report K as the number of studies in the analysis, N as the number of effect sizes included, β0 as the value of the intercept, β1 as the value of the slope associated with a certain moderator, the associated standard errors, σu2 as the variance associated with the study level, σe2 the variance of the random error, and the −2*loglikelihood values.
.18 (.12)	Reference	−.25 (.32)	−.13 (.30)
−.07 (.30)	.25 (.32)	Reference	.12 (.39)
.05 (.27)	.13 (.09)	−.12 (.39)	Reference
		σu2 = 2.798 (.16)
		σe2 = .27 (.02)
K = 44	N = 599		−2*loglikelihood = 2343.96

When the intercept was set to vary at the study level, the average effect size decreased to .13 (.11). The between study variance of .24 (.10) was significant for p < .05 and the residual variance decreased but was still significant with 2.79 (.16).

Given the highly significant study level and residual variances, we carried on by including each of our moderators in the multilevel model. Compared to the results obtained in the classical metaregression, none of the continuous moderators had a significant impact on the summary average effect size estimation. The slope for the quality of the studies was .05 (.03), for age .01 (.01), and for time since treatment −.04 (.04). The average effect sizes for the types of diagnoses were .18 (.12) for breast cancer, −.07 (.30) for testicular cancer, and .05 (.27) for other diagnoses. The slopes had different orientations depending on the reference dummy-coded category. For all these results both between study and residual variances continued to be significant and of roughly similar values (see Tables 3 and 4 for details).

Publication Bias Analysis

We performed publication bias analyses for each cognitive function examined, and some analyses need to be treated with caution. When analyzing the data irrespective of the type of control group, the intercept had a p < .10 only for capacity of attention. In cross-sectional studies, verbal immediate free recall, delayed memory, focused attention, executive and motor functions, were influenced by publication bias. In longitudinal studies, the bias was present for immediate free recall, arithmetic, verbal free recall immediate, and visual free recall immediate. In the baseline subgroup, memory, executive functions, spatial, and verbal abilities were lower than .10 significance values, while the analyses of controls in longitudinal studies showed no influence of publication bias. Thus, these cognitive functions may have been reported more often in the literature if they showed impairments, while accounts of increases or lack of change might have been underreported.

Discussion

Our meta-analysis summarizes the findings from 44 studies examining an array of cognitive functions in adult cancer patients. Our primary objective was to identify which functions are impaired in each type of design. To that end, we divided studies into subgroups based on their design and calculated summary effect sizes for each type of cognitive function. The secondary objective was to identify potential factors that might explain the variability of results in previous literature. Thus, we analyzed the impact of four moderating factors on all effect sizes.

When analyzing all data, regardless of design and type of control group, only selective attention was impaired. Compared to previous meta-analyses, our effect sizes either did not reach statistical significance, or were very close to zero. Thus, pooling together a higher number of primary studies with inconsistent results, ultimately summed up to heterogeneous summary effect sizes, which did not distinguish between types of impairments (Rosenthal & DiMatteo, 2001).

Despite the influence of several confounding variables on the effect sizes, when patients were compared to controls at posttreatment, we observed small to moderate effect sizes, suggesting impairments in all aspects of verbal memory (immediate free recall, delayed free recall, and delayed recognition), in selective attention, and attention capacity. Furthermore, results in the cross-sectional subgroup analyses had lower heterogeneity values than those in other subgroups. This may suggest that cross-sectional studies are influenced by less and more similar sources of heterogeneity, while the latter may be more diverse in longitudinal studies. Irrespective of which hypothesis is true, they both suggest the need for better controlled studies to reduce subsequent heterogeneity (i.e., through proper participant matching).

Our cross-sectional analysis examining patients and controls at baseline investigated patients’ performance before exposure to chemotherapy. Patients performed strikingly better than matched controls, for instance in their verbal (g = 3.94) and spatial abilities (g = 2.34).

The difference between the baseline and posttreatment results in patients and controls is depicted in Figure 7. This pattern contradicted our expectations based on previous literature. First, at baseline in cross-sectional studies, we would have expected patients to either perform worse than controls if the cancer itself would have had a deleterious impact (Ahles et al., 2008), or at the same level as controls if participants had been matched accordingly on their educational and intelligence levels. This effect is not visible in our results. On the one hand, the high effect sizes at baseline clearly suggest that patients are not impaired before chemotherapy. They also question whether there were other factors that differentiated patients from controls from the start of the study, such as different education, socioeconomic status, different intelligence levels, and even different motivation to perform well in testing. We note, however, that the two sets of analyses are not directly comparable, thus this is a relative comparison with the results one would expect based on previous literature reviews (Vardy et al., 2008; Zachariae & Mehlsen, 2011). Posttreatment analyses are drawn mostly from cross-sectional studies, while the baseline analyses are only drawn from longitudinal studies in which patients were compared to control participants at baseline, and then at several time points postchemotherapy. Thus, the potential poorer participant matching in the baseline assessments might stem from variability in longitudinal studies.

In longitudinal studies, patients performed better in follow-up evaluations than at baseline, with small to moderate effect sizes across multiple functions. Improvements between the first and second tests were also observed in control participants, for the limited set of cognitive functions we were able to analyze. These were lower and based on a smaller number of studies than the effects computed on patients, but were less heterogeneous than the patient analyses. Just as in the two cross-sectional subgroup analyses, the lower heterogeneity may be explained by the fact that the two sets of data were sourced from different articles, which may have been affected by confounders differentially. Despite this, the effect sizes of the patient group are still high, specifically for immediate free recall, and verbal immediate free recall.

The pattern of impairments in cross-sectional studies appears congruent with previous imaging studies. Compared to matched controls, breast cancer patients had significant left lateralized white matter decreases in the parahippocampal gyrus (de Ruiter et al., 2011; Inagaki et al., 2007; McDonald, Conroy, Ahles, West, & Saykin, 2010), and reduced activations in the left lateral posterior parietal regions and left dorsolateral prefrontal cortex (de Ruiter et al., 2012, 2011; Deprez et al., 2011). The moderate impairments in selective attention may be associated with the decreased white matter in the superior fronto-occipital fasciculus and superior and medial frontal gyri observed in other imagining studies (de Ruiter et al., 2012, 2011; Inagaki et al., 2007; McDonald et al., 2010; Silverman et al., 2007). Despite the legitimacy of these hypotheses, more studies are needed in order to determine if these are the actual structural and functional changes. Furthermore, the pattern of deficits in attention and memory makes it difficult to conclude whether the memory problems exhibited by cancer survivors are dependent on damage to the medial temporal lobes, or secondary to damage in frontal or parietal neocortical regions associated with attention performance. We hypothesize that some aspects of the memory impairments may be primary and others dependent on attention deficits, but further research is needed to address this issue.

The cognitive performance increases in patients in longitudinal studies are surprising, but may be linked to either additional sources of bias or genuine long-term improvements. Patients may be prone to a relatively stronger influence of practice effects due to certain characteristics that may increase their motivation to take part in such studies, compared to control participants. Reasons could include the desire to perform well in the test, preexisting knowledge that chemotherapy may be associated with cognitive impairment, and differential setting and framing of the tests for patients and control participants (Schagen, Das, & Vermeulen, 2012). However, some deficits may only be short-lasting effects (McDonald et al., 2010; Silverman et al., 2007), or some participants may have higher cognitive reserves allowing the development of compensatory strategies in specific cognitive tasks (Ahles et al., 2010). Both these hypotheses warrant further investigation.

The multilevel moderator analyses showed that neither of our chosen moderators explained the variance in effect sizes. This is a surprising result, given that it is unlikely that a different age, or a different time since treatment would not affect cognitive functioning. Some of our hypotheses regarding this nonsignificant result are that the multilevel model collapsed all summary effect sizes for all cognitive functions and all types of designs. If cross-sectional designs have negative summary effects, while longitudinal ones have positive summary effects, the resulting average effect size may approach zero.

A second potential explanation is the description of moderators. The quality of the studies was assessed with one of the best assessment tools recommended by the Cochrane Collaboration. However, this may have been unsuitable for the types of studies conducted in this specific field, and for the time being there are no such tools available to evaluate the quality of neuropsychological studies. It may have been necessary to differentiate between specific aspects of quality, such as the presence of tests with alternative formats, the reporting of certain test scores (i.e., full-scale IQ), and the type of matching available in primary studies. The age of participants varied between 38 and 71, with an average mean age of 51.57 (SD = 6.29). The type of diagnosis is restricted to breast cancer in 70% of the studies, and the time since treatment in years varies between zero and 10 (M = 2.09, SD = 2.52). The nonhomogenous samples in primary studies reflects the higher variance within the moderators, a reason why the nonsignificant results are expected: there are not many studies available including participants of roughly the same age, or the same time since treatment, which may have led to nonsignificant results when all effect sizes are collapsed together. We consider this aspect to warrant further examination in future meta-analyses.

Due to the limitations of our meta-analysis (majorly due to the unequal distribution of primary studies) we caution the interpretation of some of our results. First, our analyses show that the heterogeneity of our estimated effect sizes was not explained by our chosen moderators. Both between-study and residual variances remained high despite the inclusion of predictors. However, effect sizes in longitudinal studies are more heterogeneous than in cross-sectional studies, possibly due to additional factors which have not been measured, or have not been reported (e.g., impact of hormonal treatment, psychological comorbidities, etc.). While many studies report matching on age and gender, intelligence is often a factor reported through various scores, ranging from the full scale IQ, to verbal or performance IQ. If the groups were not properly matched in primary studies, that would deem any meta-analytical results less trustworthy. To reduce such a possibility in future studies, intelligence should be measured and reported through the same measures (FSIQ). Throughout our analyses we assumed the control participants and patients were matched on premorbid IQ, while this may not have been the case in all primary studies. For example, if patients’ premorbid FSIQ were higher than controls’, the observed deficits might not actually be mild, but severe for a highly functioning person. Due to this reason, the reporting of these scores in all future primary studies is warranted.

Second, the cancer diagnosis moderator was actually a proxy of treatment protocols and gender. While the type of treatment would be a valuable factor to analyze in relation to cognitive functioning, there is a high variability in the treatments reported, depending on the staging of the illness and patient-level medical characteristics. Importantly, the use of other classes of drugs such as corticosteroids, hormone antagonists or antiemetics, may also have influenced the results (Lupien et al., 2002), but these details are not always reported within the literature. Primary studies do not usually report the results of cognitive tests separately on the types of treatments.

As most studies with adults focus on patients with breast cancer, the cancer diagnosis is also a proxy of the gender. Previous studies with pediatric cancer patients have shown that female gender may be a vulnerability factor to chemotherapy-induced changes (von der Weid et al., 2003). However, the results of cognitive tests were not reported separately in any of the studies including both males and females; thus gender could be a moderator to be accounted for in future studies, especially when extending to the assessment of treatments for malignancies other than breast cancer.

Third, due to technical limitations, our review only included studies reporting means and standard deviations. Although we attempted to obtain this data from corresponding authors, this was not always possible, and resulted in 23 studies being excluded. However, the Egger’s values are only significant for four cognitive functions, thus most our analyses were not influenced by publication or selection bias. This relates to our fourth limitation—not adjusting for the publication bias found in these analyses. The distribution of our data within multiple subgroups, as well as the unaccounted between-study heterogeneity (Terrin, Schmid, Lau, & Olkin, 2003) would have resulted in inaccurate trim-and-fill results. However, there are only four specific results that should be treated with caution; specifically, in cross-sectional studies, delayed and verbal immediate memory, and in longitudinal studies, immediate memory and verbal immediate free recall.

Finally, while heterogeneity was lower in cross-sectional subgroup analyses, it remained high in longitudinal studies. Our work should assist in reducing measurement noise in future empirical work. This should help minimize heterogeneity in future meta-analyses, as well as reducing the number of confounding variables influencing results of primary studies.

Conclusions

The present paper summarizes research in the field of chemotherapy-induced cognitive impairments, while highlighting the nature, extent of impairments, and factors influencing their identification. Despite the considerable heterogeneity of the data, the results obtained from cross-sectional studies could be considered the most reliable. With potentially less influence from additional variables, patients in cross-sectional studies performed worse than controls on tests of capacity of attention, selective attention, verbal memory, immediate, and delayed, in both free recall and recognition tasks. Although the interplay between attention and memory impairments remains a matter for future research, our results suggest that the impairments might be linked to both frontal and medial temporal lobe dysfunction.

We have shown that cognitive performance prior to chemotherapy was higher in patients than in controls. That suggests that malignancy itself was not responsible for neuropsychological late effects, but it also casts doubt on the quality of participant matching and unreported sources of bias in longitudinal studies.

Our moderator analyses were not significant, which is surprising given the plethora of factors that could influence cognitive data. This is the reason why, for the aid of future analyses on this topic, we suggest a number of guidelines that could be followed in future studies:

The use of shorter neuropsychological batteries by focusing specifically on certain cognitive functions. This strategy would shorten the testing time and maintain participants’ interest active throughout the sessions. This option could decrease the differences in participants’ motivational levels during testing.

Longitudinal studies should only use cognitive tests with alternative formats to avoid practice effects. Alternatively, when this standard cannot be achieved, it would be preferable to pursue cross-sectional studies.

Avoiding the use of tests that lack equivalent alternative formats and lacking sensitivity to very mild cognitive impairments (i.e., MMSE or RBANS).

When using neuropsychological tests, striving to use very similar versions of the same cognitive tests between research groups, and reporting the same scores. This would promote a more consistent impression of the impairments across studies. Examples of tests to be used would be the HVLT (and other similar versions, such as RAVLT or CVLT), the ROCFT (or other similar versions), any sections of the DKEFS (or similarly, the Stroop, Trail Making Test, and Controlled Oral Word Associations), D2 (or Ruff 2&7), and Digit Span for working memory.

Consistently grouping test scores into cognitive functions, as the high number of neuropsychological tests makes it difficult to understand whether two different results refer to the same function. The present meta-analysis groups the tests within cognitive functions as suggested by Strauss et al. (2006).

Memory and attention have been consistently found impaired in many primary studies and meta-analyses, including our own. However, the links and mechanisms of these impairments are not yet explained. They could be investigated further through the development and administration of newly designed tests inspired by the neural mechanisms of these processes.

Reporting premorbid intelligence levels in a unitary fashion, as this is a key factor in matching controls and patients. This can be achieved by reporting the full-scale IQ score as measured with the WTAR or NART. These tests should correlate with most cognitive measures, unless the patients have very specific functional or structural brain changes due to treatment.

All cross-sectional studies should match participants closely on age, education, gender, and IQ, as all four variables could potentially change the whole interpretation of a cognitive dataset.

Results should be reported separately on the basis of moderators that could induce additional bias: gender, age of participants (if they vary between younger and older adults), relapses (as a potential factor relevant for the severity of impairments), types of diagnoses, and treatments.

Studies including control groups at baseline and follow-up could also report this set of data in the same table as the results of the patients.

While incorporating these conclusions and suggestions, future research should focus on the stability of these side effects, the link between memory and attention impairments, and the treatments and clinical vulnerability factors that may predispose some patients more to impairments, rather than others. These future findings would inform the cognitive intervention strategies required to help present and former patients cope with chemotherapy-induced cognitive impairments.