Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of "Far Transfer": Evidence From a Meta-Analytic Review.

It has been claimed that working memory training programs produce diverse beneficial effects. This article presents a meta-analysis of working memory training studies (with a pretest-posttest design and a control group) that have examined transfer to other measures (nonverbal ability, verbal ability, word decoding, reading comprehension, or arithmetic; 87 publications with 145 experimental comparisons). Immediately following training there were reliable improvements on measures of intermediate transfer (verbal and visuospatial working memory). For measures of far transfer (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) there was no convincing evidence of any reliable improvements when working memory training was compared with a treated control condition. Furthermore, mediation analyses indicated that across studies, the degree of improvement on working memory measures was not related to the magnitude of far-transfer effects found. Finally, analysis of publication bias shows that there is no evidential value from the studies of working memory training using treated controls. The authors conclude that working memory training programs appear to produce short-term, specific training effects that do not generalize to measures of "real-world" cognitive skills. These results seriously question the practical and theoretical importance of current computerized working memory programs as methods of training working memory skills.

Working memory refers to the idea that there is a general capacity-limited system (or set of systems) responsible for the storage and manipulation of information in the human mind. In the last few years, studies of working memory training have bourgeoned. In this article, we review the background on working memory training and provide a meta-analytic review of the many studies in this field. Our conclusion is that there is no good evidence that working memory training improves intelligence test scores or other measures of ”real-world” cognitive skills. We finish with some recommendations for future research.

The Foundation of Working Memory Training

The rationale for working memory training developed from suggestions that limitations in working memory capacity may have wide-reaching effects on other aspects of cognition. For example, according to one theory variations in fluid intelligence reflect to a large degree variations in working memory capacity (Engle, 2002; but see Heitz et al., 2006). Others have argued that limitations in working memory capacity may be responsible for impairments in the development of reading (Swanson, 2006), language (Archibald & Gathercole, 2006), and mathematical skills (Passolunghi, 2006). The idea that working memory limitations may place constraints on diverse higher cognitive functions leads directly to the suggestion that if working memory capacity can be increased by training, this should produce transfer effects to diverse untrained tasks that depend on such a capacity (Shipstead, Redick, & Engle, 2010).

A specific example makes the approach clearer. In a pioneering study, Klingberg, Forssberg, and Westerberg (2002) assessed whether they could improve working memory capacity by training and whether this, in turn, would produce improvements on other cognitive tasks. The participants were children diagnosed with attention-deficit/hyperactivity disorder (ADHD). Children in the working memory training group were given computerized training on three working memory tasks (visuospatial span, backward digit span, letter-span) and a choice reaction time task. For each of the working memory training tasks, the difficulty level for the trained group was adjusted adaptively across trials by changing the number of stimuli to be remembered in response to the participant’s performance. The logic of such an adaptive training regime is that it makes the task challenging and ensures that participants are performing at the limits of their working memory capacity. Participants in the trained group were required to complete 30 trials on each task each day (roughly 25 min training per day) for a total of 24 days of training, giving a total training dose of roughly 10 hr spread over 5 to 6 weeks. Participants in the control group performed easy versions of the same tasks where the difficulty level was fixed to a low level in each task. However, children in the control group only did 10 trials per session on each task and had less contact with the experimenter than the children in the treatment group.

Klingberg et al. (2002) reported that working memory training improved performance in comparison to the control group on a variety of tasks, including measures directly trained (visuospatial working memory) and tasks similar to those trained (span board, another visual memory task). More surprisingly, they also reported significant increases in scores on a standardized measure of intelligence (Raven’s progressive matrices) after working memory training. This is an example of far transfer—improvement on a task that is seemingly remote from the tasks that have been trained. The claim that playing a set of computer games for roughly 10 hr can improve a child’s intelligence test scores is provocative and has led many others to try to confirm such an effect.

This study proved highly influential (and formed the basis of the popular commercial working memory training program CogMed; www.cogmed.com). However, there are some obvious weaknesses in this study: The group sizes (7 children per group) are tiny—this study is, like many other studies in this area, severely underpowered. Another problem with the design of this study is that any differences found as a result of training may reflect differences in the duration of the computerized tasks performed by the two groups, rather than being effects of adaptive working memory training, per se. Nevertheless, if the claim that working memory training improves intelligence test scores could be confirmed and substantiated it would have potentially important implications for education and economic productivity, as well as for theories about the nature and limitations of human cognitive abilities. Because studies in this area are generally underpowered, summarizing them in a meta-analysis may be valuable as a way of clarifying the conclusions that can be drawn from the many inconsistent findings.

Prior Reviews of Working Memory Training

In this article, we report an updated meta-analysis of current evidence for the effectiveness of working memory training. We focus on two key issues: (a) Does working memory training improve the skills directly trained? (b) If so, does such training transfer to improve other skills such as intelligence test performance? Recently, there have been many working memory training studies that have been summarized in several qualitative and quantitative reviews, with seemingly conflicting conclusions (see Appendix in the Supplemental Materials available online for references to other reviews and meta-analyses of interest).

Melby-Lervåg and Hulme (2013) reported a meta-analysis of all 23 published studies (at the date of their review) of working memory training in both adults and children. They found evidence that the training programs produced reliable short-term improvements in working memory skills. However, these improvements in working memory skills did not appear to be maintained a few weeks after training had ended. Furthermore, there was no convincing evidence of generalization from working memory training to other skills (nonverbal and verbal ability, inhibitory processes in attention, word reading, and arithmetic). They concluded that working memory training programs appear to produce short-term effects that do not generalize to tasks that have not been directly trained. Clearly, the most critical issue, theoretically and practically, is whether or not working memory training produces far-transfer effects (Barnett & Ceci, 2002; Taatgen, 2013), for which our previous review (Melby-Lervåg & Hulme, 2013) failed to find any convincing evidence. These findings of small immediate effects with no reliable effects at follow-up have also been replicated by Schwaighofer, Fischer, and Buhner (2015).

Recently, two meta-analyses have claimed that working memory training can be effective in enhancing cognitive skills in adulthood (Au et al., 2014) and stemming cognitive decline in old age (Karbach & Verhaeghen, 2014). However, the conclusions from these articles can be questioned because of the following: (a) The failure to take account of baseline differences when calculating effect sizes is of great importance, because relying only on posttest differences can cause biased effect-size estimates, especially in a field with small sample sizes and studies that have imbalance at baseline; (b) the failure to base conclusions on studies that include treated control groups, because only these studies provide adequate control for nonspecific effects (such as familiarity with being assessed on a computer) that may arise in computerized working memory training studies; and (c) the merging of measures such as reading comprehension and executive measures into one far-transfer construct. Melby-Lervåg and Hulme (2016) reanalyzed the studies from these two meta-analyses and concluded that there was no convincing evidence that working memory training produces general cognitive benefits (see also Dougherty, Hamovitz, & Tidwell, 2016). However, recently there have been other meta-analyses of working memory and cognitive training concluding that it is effective for specific age/patient groups (Karr, Areshenkoff, Rast, & Garcia-Barrera, 2014; Kelly, Loughrey, & Lawlor, 2014; Spencer-Smith & Klingberg, 2015; Weicker, Villringer, & Thöne-Otto, 2016).

Previous reviews seem to show conflicting findings for two main reasons: First, some methodological decisions may have led to biased conclusions (see Melby-Lervåg & Hulme, 2016). Second, previous reviews typically have examined subsets of studies restricted to specific types of working memory training, age groups, or participant status. An alternative approach is to include as broad a range of studies as possible and then examine whether certain study characteristics can explain variations between them. This is the approach adopted here. We have identified a total of 87 publications with 145 different experimental comparisons (up from 23 publications with 30 experimental comparisons identified by Melby-Lervåg & Hulme, 2013, and from 47 studies with 65 experimental comparisons identified by Schwaighofer et al. (2015), which is a considerably larger number than all other reviews in this field. We have designed our review to be as inclusive as possible to reach a clear statement as to what conclusions are justified by current evidence in this contentious field.

Also, in this review we will pay particular attention to possible effects of publication bias using a novel method, p-curve analysis (Simonsohn, Nelson, & Simmons, 2014). In the past few years, there has been much attention paid to issues of replicability in psychology (e.g., Ioannidis, 2012; Open Science Collaboration, 2015; Pashler & Harris, 2012). By analyzing publication bias in more detail, we can get a better understanding of how it can affect a field when one study shows a newsworthy finding with an extremely large effect size (as in the study by Klingberg et al., 2002). Our analysis using p-curves provides evidence of publication bias in studies of working memory training.

The Current Review

The current meta-analysis focuses on four interrelated questions:

Does working memory training improve performance on working memory tasks (i.e., produce improvements on the tasks trained and on visuospatial and verbal working memory tasks)?

Does working memory training improve performance on tests of nonverbal skills (particularly measures of nonverbal reasoning, such as Raven’s matrices)?

Does working memory training improve performance on tests of verbal skills (verbal ability, word decoding, reading comprehension, and arithmetic)?

Is there a relationship between intermediate-transfer and far-transfer effects (i.e., if training produces improvements on measures of working memory, are improvements on more distant measures of transfer such as nonverbal reasoning proportional to the improvements found in working memory)?

Question 1 addresses whether training is effective in improving performance on measures that are identical or closely related to tasks that have been trained (near-transfer measures). Questions 2 and 3 address aspects of far transfer; more specifically, they are relevant to perhaps the most provocative claim for working memory training—that it can increase scores on measures of intelligence and attainment. Question 4 addresses the mechanisms responsible for possible far-transfer effects. Theoretically, such far-transfer effects are only expected in the presence of improvements in working memory capacity, because effects of working memory training on fluid intelligence or academic attainment are typically seen as being mediated by (i.e., dependent on) increases in working memory capacity (Harrison et al., 2013; Lange & Süß, 2015; Tidwell, Dougherty, Chrabaszcz, Thomas, & Mendoza, 2014; von Bastian & Oberauer, 2014). If working memory training studies find increases in fluid intelligence or attainment in the absence of increases in working memory capacity, this pattern would be difficult to explain theoretically. It would have to be argued, for example, that the component of intelligence that is improved by working memory training is different from the components of intelligence that are dependent on working memory capacity.

The four questions outlined above are critical for any attempt to assess evidence for the effectiveness of working memory training. However, in reaching a satisfactory answer to these broad questions, a number of methodological issues need to be taken into account. One potentially critical factor is the type of activity to which the control group is exposed (Boot, Simons, Stothart, & Stutts, 2013; Buschkuehl & Jaeggi, 2010; Melby-Lervåg & Hulme, 2013; Shipstead, Hicks, & Engle, 2012). We have categorized studies into those with untreated controls (i.e., where the control group receives no intervention) or treated controls (i.e., where the control group receives a non-working memory training intervention of a similar type and of equivalent intensity and duration). Arguably, only studies with treated control groups can provide convincing evidence of specific benefits from working memory training. If a study only has an untreated control group, any effects of training could arise from quite general effects—for example, familiarity with computerized tasks, additional contact with the experimenter (demand characteristics), and motivational differences related to belief about the study purpose (expectancy effects) rather than effects on working memory processes, per se. Because untreated control conditions are likely to overestimate the true size of any training effect, they are potentially most useful for preliminary evaluations of novel effects, but not for understanding causal mechanisms or evaluating whether a treatment offers an improvement over current practice (see Mohr et al., 2009, for a review). Thus, if there are true effects of training, this should be apparent both in studies using treated controls and in studies using untreated controls. For these reasons, we will analyze studies with untreated controls separately from those with only treated controls, and our discussion and interpretation of the meta-analytic results will largely focus on analyses of studies with treated controls.

Studies of working memory training also differ on numerous other dimensions that potentially may affect the results of training. Therefore, we will conduct analyses to examine whether training effects differ according to:

The age of participants. On the basis of ideas about brain plasticity (Buschkuehl, Jaeggi, & Jonides, 2012), some researchers have suggested that individuals at different ages may be more receptive to benefits from working memory training (Lövdén, Bäckman, Lindenberger, Schaefer, & Schmiedek, 2010). This finding could take the form of working memory training being more effective for children or young adults relative to older adults, although other research (Zinke et al., 2014) indicates that age may moderate working memory training gains even within the older population.

The duration or dose of training received. One of the first working memory training studies (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008) reported that the amount of transfer to fluid intelligence was dependent on the number of training sessions completed. Working memory training studies have varied widely in terms of training duration, which may account for some of the variation in transfer observed between studies.

Learner status. Under the assumption that individuals who have cognitive impairments (e.g., ADHD, dyslexia, low working memory, or other neuropsychological disorders) may be more receptive to benefits from working memory training, we separated studies according to whether participants were selected for having a learning difficulty. Statistically, individuals with lower cognitive abilities before training may have more room for improvement, and thus training and transfer might be more effective for these individuals relative to healthy, typically developing participants.

Study design. We examined whether studies used random allocation of participants to conditions. Traditionally, it has been argued that random allocation of participants to conditions is the surest way of ensuring that the results of an intervention reflect a causal effect (Shadish, Cook, & Campbell, 2002). However, meta-analyses indicate that under some circumstances, data from nonrandomized experiments may yield similar estimates of effect size to those from randomized experiments (e.g., Heinsman & Shadish, 1996; Shadish & Ragsdale, 1996). The analyses presented later will examine this issue further.

Publication bias. Several studies have found that intervention studies are particularly vulnerable to publication bias and for lacking replicability (Cuijpers, Smit, Hollon, & Andersson, 2010; Scherer, Langenberg, & von Elm, 2007). Here, we will pay particular attention to this, both by retrieving as much grey literature as possible (see Rothstein, Sutton, & Borenstein, 2005) and by estimating the impact of publication bias using statistical procedures. Studies from the grey literature were retrieved through electronic database searches, by e-mailing researchers in the field, and by attending conferences and asking for posters/unpublished material (see Fig. 1).

Fig. 1.

Flow diagram for the search and inclusion criteria for studies in this review.

Type of working memory training program. Numerous training programs have been used in working memory training studies. For example, the commercial CogMed program includes various versions of verbal and visuospatial memory span tasks. Other commonly used working memory training programs include: (a) running memory span tasks, where participants must recall in order only a specified number of items at the end of a long list of stimuli; and (b) complex memory span tasks, where the participant completes a distractor processing task interleaved with to-be-remembered stimuli within the span task. Finally, variations of the N-back task have been used frequently; in these tasks participants indicate whether or not the currently presented stimulus matches one that was presented n stimuli back in a list (e.g., Jaeggi et al., 2008). The different types of working memory training tasks used could account for the variation in results across studies and is examined here as a potential moderator. In addition, the stimulus content (verbal, visuospatial, or both) of the working memory training programs was coded.

In summary, we present a meta-analysis to synthesize evidence from all studies of working memory training (both published and unpublished) that we have been able to identify. We will focus particularly on evidence for the most provocative claim of working memory training (that it improves intelligence), though we will also consider other claims, particularly those concerning generalized improvements on untrained measures of both verbal and visuospatial working memory. This study will clarify previous working memory training meta-analyses. We will provide more fine-grained analyses of several different outcomes, analyzing follow-up effects and controlling for baseline differences, while examining multiple working memory training programs (in contrast to Au et al., 2014) and including participants across the entire life span (in contrast to Au et al., 2014, and Karbach & Verhaeghen, 2014). In contrast to Melby-Lervåg and Hulme (2013) and Schwaighofer et al. (2015), we have a considerably larger sample of studies, providing us much more robust estimates of effect size and allowing us to perform moderator analyses that have more power (Hedges & Pigott, 2004). Further, in comparison to Melby-Lervåg and Hulme (2013) and Schwaighofer et al. (2015), we will also provide a more fine-grained analysis of working memory measures and publication bias. Critically, we will use mediation models to examine whether differences among the studies in terms of gains in working memory are related to differences in gains on the transfer measures.

Method

This meta-analysis was designed in line with the statement for systematic reviews developed by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, www.prisma-statement.org).

Search, inclusion criteria and coding

The literature search, criteria for inclusion and exclusion and flow of studies are shown in Figure 1. Only studies that had a control group and a training group with pretest and posttest measures before and after working memory training were included in the review. Our decision as to whether a study trained working memory was guided by previous research examining the factor structure of working memory measures (see Ackerman, Beier, & Boyle, 2005; Engle, Tuholski, Laughlin, & Conway, 1999; Gathercole, Pickering, Ambridge, & Wearing, 2004; Kane et al., 2004; Schmiedek et al., 2009). For a study to be included, the training had to involve tasks that typically have been found to load on working memory in latent-variable studies. Examples include visuospatial and verbal versions of simple span (forward and backward), complex span, running span, updating (e.g., keep track), and N-back (single and dual). In cases where an intervention had multiple components, the working memory tasks had to constitute at least 50% of the intervention. Studies based purely on training task-switching, inhibition, or reasoning were excluded in an attempt to isolate the construct being trained as working memory and not other potentially related constructs (e.g., executive functioning). We included only studies of computerized working memory training.

The studies examined could be randomized or quasi-experiments, but they had to include tests of either nonverbal ability, verbal ability, reading comprehension, word decoding, or arithmetic as outcome measures. We also included verbal and visuospatial working memory outcomes, but only if the study included one of the far-transfer constructs. Measures of problem solving without a clear reliance on language were coded as nonverbal ability tests (common examples include Raven’s progressive matrices and the Cattell Culture Fair test). Measures of verbal ability were largely tests of receptive or expressive vocabulary knowledge, along with reasoning based on alphanumeric content (e.g., Letter Sets, Number Series). Measures that involved text reading with subsequent questions about the meaning of a passage were classified as reading comprehension tests. Measures of word decoding included tests of the accuracy or fluency of word or nonword reading. Arithmetic measures included tasks involving addition, subtraction, multiplication, or division. Near-transfer measures were tests that were similar or identical to the tasks trained. For working memory measures, we distinguished between measures of verbal and visuospatial working memory (where the participant had to remember verbal or visuospatial material, respectively). Simple memory span tests such as digit span were excluded from the analyses if only the forward version was administered, unless the forward-only version of the task was considered a criterion measure that was identical to a training task. If the backward version of a simple span task was administered separately, or a combined score for the forward and backward versions was reported, the outcome was included as well.

We separated the measures into three different categories: (a) near-transfer measures (tests that were similar or identical to the tasks trained), (b) intermediate-transfer measures (verbal and visuospatial working memory measures), and (c) far-transfer measures (measures that differed substantially from those trained, i.e. nonverbal ability, verbal ability, reading comprehension, word decoding, or arithmetic).

We also took steps to account for the nonindependence of effect sizes from the same or different studies. Violating the assumption of independence by computing an overall effect size based on information from the same sample more than once can lead to biased estimates (Borenstein et al., 2009). For this reason studies from the same author were examined to detect duplicate samples, and studies based on the same participants were only coded once (see Fig. 1 for detailed information). When a study had multiple indicators for the same construct (for instance more than one measure of verbal working memory) the mean of the indicators was used to yield a single effect size for that study. Finally, some studies compare the same control group to different experimental groups and are included in the same analysis of a mean effect size for treated and untreated controls. Because the correlations between the multiple comparisons and their outcomes are not reported in the original studies reviewed, we included these studies in the analyses, assuming zero correlation between the outcomes. Note, however, that we also did analyses where we combined the different treatment groups into one in these studies, and this produced essentially identical results to those reported here.

All the studies were coded by two independent raters. The interrater correlation (Pearson’s) for outcomes was r = .99, 95% CI [.97, 1], p < .0001, and agreement rate = 97.00%. For the moderators, the agreement rate was 98%. Any disagreements between raters were resolved by consulting the original article or by discussion.

Meta-analytic procedure and analysis

The analyses were conducted using the “Comprehensive meta-analysis” program (Borenstein, Hedges, Higgins, & Rothstein, 2005). We calculated effect sizes by dividing the differences in gain between pre- and post-test in the treatment and the control group by the pooled standard deviation for each group at pretest; this method of effect-size calculation for pretest-posttest designs is recommended (Morris, 2008). Thus, when the effect size is positive, the group receiving working memory training made greater pretest-posttest gains than the control group. We adjusted the effect size for small samples using Hedges’s g (Hedges & Olkin, 1985). Effect sizes for follow-up tests were calculated in an analogous way (pretest to follow-up).

The mean effect sizes were calculated by a weighted average of individual effect sizes using a random-effects model. Because previous meta-analyses have found a large difference between studies using treated and untreated controls (Dougherty et al., 2016; Lilienfeld, Ritschel, Lynn, Cautin, & Latzman, 2014; Melby-Lervåg & Hulme, 2013), these two designs were analyzed separately. Treated controls received computerized training either based on nonadaptive memory tasks or other tasks that did not involve memory training but which were of similar duration to the training received by the intervention group. Untreated controls had no contact other than completing the pretest, posttest, and follow-up transfer sessions.

For moderator analyses, studies were separated into subsets based on the categories in the categorical moderator variable (e.g., children vs. young adults vs. older adults). A Q test was used to examine whether the effect sizes differed between subsets. When there were fewer than four studies in a subset (k < 4), this analysis was not conducted. The overlap between confidence intervals was used to examine the size of the difference between subsets of studies. Because of the limited number of training studies examining follow-up effects, moderator analyses were not conducted. In addition, moderator analyses were not calculated for decoding, reading comprehension, and arithmetic, given the limited number of comparisons for treated and untreated controls.

Publication bias refers to the notion that a mean effect size can be upwardly biased because only studies with large or significant effects get published (i.e., file-drawer problem with entire studies) or that authors only report data on variables that show effects (often referred to as p hacking, or the file-drawer problem for parts of studies; see Simmons, Nelson, & Simonsohn, 2011; Simonsohn et al., 2014). In line with recommendations for meta-analyses, we made special efforts to retrieve studies from the grey literature and used this as a moderator when possible (Higgins & Green, 2011). To estimate the impact from publication bias statistically, commonly funnel plots have been used in combination with a trim-and-fill analysis. However, there are several problems with the funnel plot/trim-and-fill method (Lau, Ioannidis, Terring, Schmid, & Olkin, 2006). P-curve is a recently developed method that deals with the weaknesses in the funnel plot/trim-and-fill analysis (Simonsohn et al., 2014). A p-curve plots the distribution of statistically significant p values (p < .05) in published studies, and the shape of the p-curve is a function only of the effect size and sample size, when the power level is taken into account. If there are true effects, one expects the distribution of published p values to be right-skewed with more low (.01) than high (.04) p values. However, if a set of studies is affected by publication bias (because researchers discard entire studies or discard analyses or parts of studies), the p-curve becomes left-skewed or flat. Such a form of p-curve is said to provide no evidential value (i.e., no support for an appreciable effect size).

When coding articles, it became clear that there were numerous instances of missing data. If data were critical to calculate an effect size, articles with missing data were excluded if authors did not respond to an e-mail request to provide the data (see inclusion criteria in flow chart). In cases where an effect size could be computed on one outcome but data were missing on other outcomes or moderator variables, the study was included in all the analyses for which sufficient data were provided.

Moderator variables

Moderators are variables that may explain why different studies show different results (Pigott, 2012). The following moderator variables were used:

Age

The average age of participants in each study was coded. Because of a nonnormal distribution, it was not possible to analyze age as a continuous variable. Studies were therefore separated into three groups: studies of children (≤18 years), adults (18–64), and older adults (≥65 years).

Training dose

The duration of training (total number of hours in training) was coded. Again, because of a nonnormal distribution, training duration was divided into discrete bands (studies with a total training duration of up to 10 hr vs. those with more than 10 hr).

Design type

The procedure for separating participants into training and control groups was coded (randomized or nonrandomized).

Learner status

The sampling of participants in the study was coded: learning disorder (e.g., ADHD, reading, math, or other learning disorders) or unselected.

Training type

The training programs were split into four categories: N-back, CogMed, complex span, and other tasks (which could include combinations of the four specific types of training listed). There were too few running span studies to consider them as a separate category.

Publication type

Each experiment was coded as grey literature (theses, dissertations, conference posters) or published studies (journal articles, chapters, peer-reviewed conference proceeding papers).

Results

Characteristics concerning the studies included in the review are shown in Table S1 in the Supplemental Material available online. This table shows sample age, sample size, outcome measures, and effect sizes for each time point (posttest or follow-up) for each comparison within each study. Table S2 in the Supplemental Material available online shows how each study was coded on the moderator variables. Sample size and mean sample size for all studies included in each analysis are shown in Table S3 in the Supplemental Material available online. For forest plots for each outcome, see Figures S1 to S8 in the Supplemental Material available online. The full dataset is provided in an Excel sheet in the Supplemental Material available online.

Immediate effects of working memory training on far-transfer measures

Figure 2 shows a summary of the effects of working memory training on far-transfer measures. In studies with treated controls, the effects of training on nonverbal ability, verbal ability, word decoding, and arithmetic are close to zero and not significant (see Table 1). For reading comprehension, the effect size for studies with treated controls is small though significant (g = 0.15). Closer inspection shows that 10 comparisons with treated controls give small to large positive effects (g = 0.10 to 0.87). Of these, six comparisons across four studies (Jaeggi, Buschkuehl, Shah, & Jonides, 2014; Lee, 2014; Redick & Wiemers, 2015; Shiran & Breznitz, 2011) show an odd pattern with the control group showing decreases in reading comprehension from pretest-posttest (see Redick, 2015, for examples of this pattern). This pattern of results makes these studies very difficult to interpret. Theoretically, there is no reason to expect decreases in a relatively stable construct such as reading comprehension ability between pretest and posttest in a control group; such decreases presumably can only reflect error of measurement. Also, one of these studies (Shiran & Breznitz, 2011) showed large effects compared with the others (above 3 SDs from the mean effect size). When the comparisons with the problematic pretest-to-posttest decline in the control group are excluded, the effects of working memory training compared with treated controls for reading comprehension becomes trivial (g = 0.08) and similar in magnitude to the other far-transfer outcomes.

Fig. 2.

Mean effects (g) on the transfer measures for studies with treated and untreated controls (k = number of studies)

Table 1.

Effects of Working Memory Training Compared to Treated and Untreated Control Groups at Immediate Posttest

Construct	Comparisontype	Mean effect size (g)[95% CI]	No. of studies	Q	I²	τ²
Nonverbal abilities	Training vs. treated controls	0.05 [−0.02, 0.13]	67	55.37	0	0
	Training vs. untreated controls	0.20 [0.11, 0.28]**	53	59.34	12.36	0.01
Verbal abilities	Training vs. treated controls	0.05 [−0.07, 0.17]	22	13.91	0	0
	Training vs. untreated controls	0.03 [−0.09, 0.14]	16	11.15	0	0
Word decoding	Training vs. treated controls	0.08 [−0.09, 0.24]	10	1.93	0	0
	Training vs. untreated controls	0.01 [−0.16, 0.17]	7	2.31	0	0
Reading comprehension	Training vs. treated controls	0.15 [0.03, 0.27]*	19	12.84	0	0
	Training vs. untreated controls	0.12 [−0.07, 0.31]	7	7.71	22.17	0.01
Arithmetic	Training vs. treated controls	0.06 [−0.08, 0.19]	15	7.57	0	0
	Training vs. untreated controls	0.12 [0.01, 0.23]*	14	6.77	0	0
Verbal working memory	Training vs. treated controls	0.31 [0.19, 0.42]**	60	123.79**	52.34	0.10
	Training vs. untreated controls	0.42 [0.24, 0.61]**	38	130.64**	71.68	0.24
Visuospatial working memory	Training vs. treated controls	0.28 [0.16, 0.40]**	40	60.51	35.54	0.05
	Training vs. untreated controls	0.51 [0.34, 0.69]**	25	62.82**	61.79	0.11
Criterion measure	Training vs. treated controls	0.80 [0.62, 0.97]**	22	41.20**	49.03	0.08
	Training vs. untreated controls	1.88 [1.33, 2.42]**	16	130.47**	88.50	1.06

Note: *p < .05; **p < .01.

As seen in Table 1, even studies using passive controls did not produce significant effects on verbal abilities, decoding, and reading comprehension. Thus, when directly comparing treated and untreated controls, the difference in effect size was only statistically significant in the case of nonverbal ability, Q(1) = 6.09, p = .01. On most measures the pattern of findings across studies were consistent (the true variation between studies was zero or not significant, see Table 1). Note that N-back training shows a significant effect on nonverbal ability (g = 0.15, p = .02) in studies with treated controls. Further examination of the studies with the largest effect sizes for N-back training transferring to nonverbal ability revealed several shortcomings, which unfortunately are common in the working memory training literature (see Redick, 2015). For the five largest effect sizes, all comparisons had (a) small sample sizes (less than the minimum 20 observations per group as recommended by Simmons et al., 2011) and (b) used only one test to measure nonverbal ability (in contrast to multiple indicators of the intended nonverbal ability construct; Shipstead, Redick, & Engle, 2012; von Bastian & Oberauer, 2014). In addition, four of the five comparisons had large, unexplained pretest-to-posttest decreases for the control group, which were larger than the training group’s pretest-to-posttest increases. These crossover interactions thus artificially produced large effect sizes (Boot, Blakely, & Simons, 2011; Fischer-Baum, 2015; Redick, 2015; Redick & Webster, 2014), and are responsible for the significant effect of N-back training and transfer to nonverbal ability. After only the most problematic study has been removed from the analysis (Schweizer, Hampshire, & Dalgleish, 2011), the effects of N-back training on nonverbal ability with treated controls is negligible (g = 0.10). In the moderator analyses, studies with adults showed a small though significant effect (g = 0.10) as did studies with a small training dose (g = 0.13); however, both of these significant effects within the moderator analyses include the problematic comparisons described above, and when these studies are excluded the effect sizes reduce to trivial levels. In any case, these effect sizes are arguably too small to be relevant to educational practice (Cooper, 2008; also see Promising Practices network at www.promisingpractices.net and the What Works Clearinghouse at www.w-w-c.org). For further details of the moderator analyses of far transfer, see Table S4–S5 in the Supplemental Material available online.

Follow-up effects from working memory training on far-transfer measures

Table 2 shows the effect sizes for each of the far-transfer measures at follow-up (on average, 5 months after training). For treated controls there are no statistically significant effects on nonverbal ability, verbal ability, word decoding, or reading comprehension. There is, however, a significant effect at follow-up for treated controls on arithmetic. Unfortunately, once again this effect seems to be driven by three comparisons across two studies (Alloway, Bibile, & Lau, 2013; Nussbaumer, Grabner, Schneider, & Stern, 2013), where the control group shows decreases in performance between pretest and posttest. When these studies are excluded the effect size is negligible (g = 0.14). The findings are consistent between studies; there is no significant variation between studies on any measure.

Table 2.

Effects of Working Memory Training Compared With Treated and Untreated Control Groups at Delayed Posttest

Construct	Comparisontype	Mean effect size (g)[95% CI]	No. of studies	Q	I²	τ²
Nonverbal abilities	Training vs. treated controls	−0.05 [−0.21, 0.11]	12	3.40	0	0
	Training vs. untreated controls	0.03 [−0.22, 0.28]	7	3.88	0	0
Verbal abilities	Training vs. treated controls	0.24 [−0.12, 0.60]	3	2.03	1.42	0
	Training vs. untreated controls	0.61 [−0.84, 2.06]	2	11.15**	91.03	1.00
Word decoding	Training vs. treated controls	0.02 [−0.29, 0.33]	3	1.18	0	0
	Training vs. untreated controls	−0.07 [−0.72, 0.58]	2	2.97	66.38	0.15
Reading comprehension	Training vs. treated controls	−0.09 [−0.78, 0.60]	1	—	—	—
	Training vs. untreated controls	−0.15 [−0.48, 0.18]	2	0.14	0	0
Arithmetic	Training vs. treated controls	0.22 [0.04, 0.40]*	10	5.98	0	0
	Training vs. untreated controls	0.08 [−0.15, 0.31]	6	4.99	0	0
Verbal working memory	Training vs. treated controls	0.28 [−0.004, 0.56]	10	23.32**	61.41	0.13
	Training vs. untreated controls	0.55 [0.27, 0.83]**	13	32.66**	63.26	0.16
Visuospatial working memory	Training vs. treated controls	0.40 [0.07, 0.73]*	9	21.89**	63.46	0.16
	Training vs. untreated controls	0.39 [0.02, 0.77]*	9	32.16**	75.12	0.24
Criterion measure	Training vs. treated controls	0.99 [0.57, 1.41]**	2	0.80	0	0
	Training vs. untreated controls	1.16 [0.83, 1.49]**	5	1.87	0	0

Note: *p < .05; **p < .01.

Immediate effects of working memory training on intermediate-transfer measures

Figure 2 shows a summary of the effects from working memory training on verbal and visuospatial working memory measures. These effects are significant and moderate in size, and there is evidence of true heterogeneity between studies (see Table 1). On verbal working memory tests (see Table S6 in the Supplemental Material available online), children and older adults show significantly larger effects of training than adults, and samples with learning disorders show larger effects than samples without learning difficulties. Also, for verbal working memory, CogMed shows a significantly larger effect than the other training programs (see Table S6 in the Supplemental Material available online). For visuospatial working memory, none of the moderators for treated controls were reliably related to variation between studies (see Table S7 in the Supplemental Material available online), although studies with treated controls produced smaller effects compared to untreated controls, Q(1) = 4.52, p = .03.

Follow-up effects of working memory training on intermediate-transfer measures

Table 2 shows the effect sizes for each of the intermediate-transfer outcomes at follow-up (on average, 5 months after training). At follow-up, the effects for verbal working memory were no longer significant in studies with treated controls but still significant for studies with untreated controls. For visuospatial working memory, there were still significant effects at follow-up (see Table 2). It is important to note, however, that for many studies, the visuospatial working memory tests consisted of tasks that were similar in stimuli or method to those that were trained (e.g., the CogMed program and the span tests used in many of the CogMed studies).

Effects of working memory training on near-transfer measures

There were large effects on tasks that are similar or identical to those that are trained (see Fig. 2). The findings, though statistically significant, vary across studies (true heterogeneity is present, see Table 1) and are maintained at follow-up (5 months after training; see Table 2). At posttest, studies with treated controls show significantly smaller effects than those with untreated controls, Q(1) = 13.70, p = .0001. In treated-control studies, age was a significant moderator, with smaller effects among children than younger and older adults. In addition, near-transfer effects were larger in subjects that were healthy and did not have a condition associated with impaired working memory. There were no other significant moderators (for details, see Table S8 in the Supplemental Material available online).

The relationship between measures of near and far transfer

A series of mediation models were used to assess the extent to which improvements in working memory performance following training accounted for improvements on far-transfer measures (nonverbal ability and verbal ability). The first model contains the subset of studies that reported measures of both nonverbal ability and working memory measures similar to the tasks that were trained. Because treated controls showed significantly smaller training effects on nonverbal ability than untreated controls, we ran a meta-regression model (random effects model, method of moments) where we controlled for the type of control group with a dummy variable (coded as 1 for treated and 0 for untreated). The results showed that type of control was a predictor of gains on tests of nonverbal ability (β = .27, p = .04). However, improvements in nonverbal ability were not significantly related to improvements on the near-transfer working memory tasks (β = −.01, p = .86, k = 26). Second, we examined whether improvements on visuospatial working memory mediated the effects from working memory training on nonverbal ability. The meta-regression model showed that type of control (untreated vs. treated controls) was a significant predictor of gains in nonverbal ability (β = .20, p < .01) but improvements in the visuospatial working memory measures did not explain any further variance (β = .13, p = .10, k = 38). Finally, we examined whether the degree of verbal working memory improvement mediates the degree of improvement on measures of verbal ability. We ran a meta-regression model where we first controlled for type of control group in the subset of studies that reported data on both verbal working memory and verbal ability. The results showed that type of control group did not account for any statistically significant differences in gains in verbal abilities (β = −.03, p = .75). Second, we examined whether improvements on verbal working memory mediated the effects from working memory training on verbal abilities, and there was no evidence of an effect (β = .11, p = .19, k = 29).

Overall, these analyses fail to provide support for the idea that improvements on measures of far transfer are mediated by improvements in working memory capacity. The absence of such effects calls into question the theoretical rationale for the training studies reviewed here, because such studies are predicated on the notion of a mediated relationship (that improvements on far-transfer measures are causally dependent on the degree of improvement in working memory capacity, which is seen as a limiting factor for performance on measures of far transfer). In contrast, the meta-regression results provided additional evidence that the type of control group used (treated vs. untreated) accounts for significant variance in the nonverbal ability outcomes.

Analyses of publication bias

To address publication bias, we analyzed whether there were differences between published studies and the grey literature (see Table S9 in the Supplemental Material available online). Overall, there was a tendency for studies in the grey literature to have smaller effect sizes, but this was only significant for verbal working memory and visuospatial working memory (note that we did not conduct tests for outcomes where there were four or fewer grey studies).

We did a p-curve analysis of published studies that have reported significant (p < .05, two-tailed) far-transfer effects from working memory training. In a p-curve analysis, if the studies have evidential value, we expect to find a right-skewed curve with more low p values than higher p values. If the opposite is the case, and the studies have no evidential value (likely due to publication bias or p-hacking), we expect to find a left-skewed p-curve with more higher than lower p values. Because of the differences between treated and untreated controls found in the previous analyses, we did one p-curve for studies with untreated controls and one for treated controls. For studies with untreated controls we expected to find evidential value (a right skewed p-curve with more lower p values), because these studies are likely to show evidential value because the untreated control comparison overestimates the “true” effects of intervention. For the studies with treated controls, we expected to find no evidential value in the p-curve because the overall mean effect size concerning far transfer from studies that use active controls is close to zero. In spite of this, numerous published studies have reported significant far-transfer effects. We therefore expect to find evidence of publication bias here (exclusion of studies or analyses).

Detailed rules for including studies in the p-curve analysis are shown in Table S10, in the Supplemental Material available online. For details of the studies that the p-curves are based on, as well as excluded studies, see Table S11 in the Supplemental Material available online. Unfortunately, there has been an extensive use of one-tailed significance tests in published studies in this field (multiple studies with analysis of variance and analysis of covariance models, which is not possible given that the F distribution is asymmetric). In the p-curve analysis these p values were transformed into two-tailed values. Six of the studies that claimed significant far transfer when using a one-tailed test exceeded p = .05 when we used this more conventional two-tailed approach (see Table S11 in the Supplemental Material available online). Because a p-curve analysis only includes studies that report significant findings, these six studies were excluded from the p-curve analysis.

Figure 3 (panel A) shows the p-curve for published studies with untreated controls. In line with our hypothesis, the p-curve shows that these studies have evidential value with a right-skewed p-curve (i.e., significantly more p values of .01 or below than higher p values, z(16) = −3.41, p < .01). Figure 3 (panel B) shows the p-curve for studies of working memory training using treated controls only; this confirms our hypothesis, as there is no evidential value from the studies of working memory training using active controls, z(11) = −2.19, p = .01. The p-curve is flat or slightly left-skewed, providing evidence of publication bias (exclusion of whole studies or parts of studies). If a real effect existed, we would expect the p-curve to be right-skewed.

Fig. 3.

p-curve analysis of studies of working memory training for articles that have tested a hypothesis of transfer effects from training to other cognitive measures. A: p-curve analysis for studies with untreated controls. B: p-curve analysis for studies with treated controls.

Discussion

Our meta-analysis of working memory training reveals a clear pattern. Current working memory training programs yield short-term improvements on both verbal and visuospatial working memory tasks. For verbal working memory, these short-term near-transfer effects are not sustained when they are reassessed after a delay of a few months. For measures of visuospatial working memory, modest training effects appear to be maintained at follow-up, but these outcome tasks often share features (memoranda, method) with the tasks trained. Most seriously, however, there is no evidence that working memory training convincingly produces effects that generalize to important real-world cognitive skills (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) even when assessments take place immediately after training, especially when compared against a treated control group.

There were two cases where there appears to be weak evidence for transfer from working memory training to measures of real-world cognitive skills: improvements in reading comprehension immediately after training (that were not sustained at follow-up) and improvements in arithmetic at follow-up (in the absence of effects at immediate posttest). However, we believe that there are strong reasons to doubt that these effects are genuine. Both effects appear to be driven by studies in which the control group show a decrease between pretest to posttest, and such a pattern inflates estimates of the effect size obtained (in extreme cases a large decrease in scores in the control group, coupled with no significant change in scores for the trained group, would lead to a significant but arguably artefactual effect of training). This pattern of declines in the controls on stable constructs such as reading comprehension and arithmetic is completely unexpected and presumably reflects measurement error. Contrary to the claims made in the original articles, we believe that such a pattern provides no evidence for a training effect (Redick, 2015). In addition, neither of these effects was significant in the untreated control comparisons.

Given our broad inclusion criteria, we were able to examine a number of possible moderators of working memory training effects on transfer. In contrast to speculation in the literature that the characteristics of the participants (age, learning difficulties) and training procedure (dose, training type) are critical for producing far transfer in certain studies, we found virtually no evidence of significant moderator effects, especially for nonverbal ability. We did find significant moderator effects on intermediate and near transfer (verbal and visuospatial working memory), suggesting that we had sufficient power to detect such effects if they were present. We believe that the lack of significant moderator effects on nonverbal ability is important since it contradicts many suggestions in the literature.

Methodological issues in the studies of working memory training

One major methodological issue comes out strongly from our review: the problem of using untreated control groups. In our analyses, we separated studies with treated controls from those with untreated controls. It is clear from our analyses that the effects of working memory training on measures of far transfer are absent (nonverbal ability, verbal ability, word decoding, arithmetic) in studies using treated control groups. As noted in the Introduction, only studies using treated control groups provide a sound basis for claiming support for specific causal effects of working memory training. Just as new medications are compared against inert pills in clinical trials to control for placebo effects, working memory training interventions should be compared against treated control groups to provide evidence for specific effects of working memory training in causing gains in unpracticed abilities. We recommend that investigators stop conducting working memory training studies with untreated control groups and that journals stop publishing them. It should also be noted that the type of active control group used is also potentially important (see Mohr et al., 2009). For example, whether an active control group uses a nonadaptive training regime or a different, adaptive activity may have effects (Weicker et al., 2016). Because many of the studies reviewed did not describe the active control group scheme in much detail, there were too few studies to do a more fine-grained analysis of this. However, this is potentially important to consider in future studies.

Another important methodological issue is the use of mediation analyses to relate changes on far-transfer measures to changes in working memory capacity. Such analyses allow us to investigate the extent to which changes on a far-transfer measure can be accounted for by changes in the theoretically critical mediating variable—working memory capacity (see Hulme et al., 2012, for an application of such analyses to explaining effects of intervention procedures in studies of reading development). In this article, we reported meta-regression models to assess the extent to which, across studies, improvements in outcome measures (e.g., verbal ability) were related to improvements in theoretically relevant mediators (e.g., verbal working memory capacity). Those analyses revealed no evidence for any mediated relationships. These findings undermine the rationale for working memory training studies. If working memory training produces far transfer because of increased working memory capacity, there should be a direct relationship between the degree to which working memory skills increase and the extent of increases in measures of far transfer, such as fluid intelligence. We believe it is important for future studies of working memory training to be adequately powered to allow for convincing tests of mediated relationships.

On a related point, many of the studies included in the meta-analyses we have reported here contain small sample sizes, which result in very low power. For instance in the 120 studies that reported transfer effects to nonverbal IQ, the mean sample size was 22.4 participants in the training group and 22.1 in the control group (see Table S3 in the Supplemental Material available online). The largest effect size for working memory training on nonverbal ability came from the earliest study (d = 2.18; Klingberg et al., 2002), which included only 7 subjects in each of the training and control groups. Given the tendency for small sample sizes to produce inflated effect-size estimates (Button et al., 2013), future studies should include larger samples to produce more precise effect-size estimates.

Here is one demonstration of how small sample sizes produce inflated effect sizes in the current dataset. Simmons et al. (2011) recommended that at a minimum, 20 subjects/observations need to be present in each cell (in this case, group). Therefore, we analyzed posttest nonverbal ability effect sizes for studies that had at least 20 subjects in each of the training and the control group versus studies that had less than 20 subjects in each of the training and the control group. For treated controls, the k = 34 studies meeting the minimum recommended sample sizes produced no effect, g = 0.01, whereas the k = 25 comparisons with fewer subjects produced a significant effect, g = 0.26. For untreated controls, the k = 31 studies meeting the minimum recommended sample sizes produced a significant effect, g = 0.16, as did the k = 18 comparisons with fewer subjects, g = 0.33. These results provide clear evidence that nonverbal ability transfer is largest in studies with untreated controls and small sample sizes, and no effects are observed in studies with treated controls and at least the minimum recommended number of subjects in the training and control groups.

The problem with studies of low power is that published studies are likely to be biased because only those with large or very large effect sizes will generate statistically significant results and therefore get published (the so-called “file-drawer problem”). This is termed by Bogg and Lasecki (2015) a “winner’s curse” because such very large effect sizes are unlikely to be true. Kraemer, Gardner, Brooks, and Yesavage (1998) have argued forcefully that meta-analyses should exclude studies that are underpowered, as this will go a long way to removing the problem of misleading conclusions arising from the file drawer problem. The p-curve analyses presented earlier provide evidence of publication bias in studies with treated control groups in this field.

Practical implications

Working memory training has frequently been claimed to increase intelligence and other important real-world skills. However, based on an analysis of the 87 studies containing 145 independent experiments reviewed here, we observed no evidence of such effects. The general pattern of a lack of transfer to real-world constructs fits with other recent meta-analyses that assess the potential therapeutic benefit of working memory training. For example, there is no evidence that working memory training reduces symptoms in individuals with ADHD (Cortese et al., 2015; Rapport, Orban, Kofler, & Friedman, 2013).

Theoretical implications

Given the strong relationship between working memory capacity and fluid intelligence (for a review, see Unsworth, 2015), the lack of transfer effects from working memory training to nonverbal and verbal abilities may appear surprising. The absence of such effects may simply reflect the fact that there is no causal relationship between working memory capacity and fluid intelligence (Harrison et al., 2013). However, working memory capacity and intelligence share approximately 50% common variance (measured with latent variables; Kane, Hambrick, & Conway, 2005). If working memory training did work to produce increases in intelligence, increases in working memory after training must be responsible for increasing the processes that are shared with intelligence. As we have shown with our mediation analyses, the available evidence suggests that is not the case—gains on measures of working memory were not related to the size of gains on measures of intelligence (nonverbal and verbal ability). This result may reflect the fact that individual differences in working memory capacity are multifaceted (Gibson & Gondoli, 2013). Having participants repeatedly practice a working memory task may not necessarily engage those aspects of working memory that reflect common processes shared with measures of intelligence. For example, according to one prominent view of individual differences in working memory capacity, individuals vary in (a) the number of items that can be held in primary memory; (b) the ability to search strategically among items in secondary memory; and (c) the ability to control attention according to goals (Unsworth, Fukuda, Awh, & Vogel, 2014). Unsworth et al. (2014) demonstrated that all three sources of variance (primary memory, secondary memory, attention control) were necessary to fully mediate the relationship between working memory capacity and fluid intelligence. An important question, then, is how likely is it that repeatedly practicing a task such as N-back leads to changes in any or all of these three sources of working memory capacity? Hypothetically, even if repeatedly practicing an N-back task were to improve an individual’s attention control, it may be that this would not be sufficient to increase the individual’s intelligence score from pretest to posttest. In addition, it is important to remember that “the variance of the score gains can have a radically different composition than the variance of the scores themselves” (Hayes, Petrov, & Sederberg, 2015, p. 9). Therefore, even though variation in working memory capacity may account for up to 50% of the variance in fluid intelligence at pretest, pretest-to-posttest increases in scores on an intelligence test do not necessarily reflect working memory increases, even after a working memory training intervention (see also Estrada, Ferrer, Abad, Román, & Colom, 2015, and Haier, 2014, for discussion of intelligence gain scores in the context of training).

As is evident in our figures, even though the meta-analytic effect sizes tended to be nonsignificant, there are certainly individual studies that demonstrate transfer effects. If working memory training is not responsible for these changes, then what is? For studies in which untreated controls are used (and even in studies with treated controls where the subjects have different expectancies than the training group about the intervention) motivation could partially explain differences, given previous research showing effects of motivation on intelligence tests (Duckworth, Quinn, Lynam, Loeber, & Stouthmaer-Loeber, 2011). For intermediate- and near-transfer effects to verbal and visuospatial working memory tasks, the development of task- or stimulus-specific strategies is likely to explain a large amount of the pretest-to-posttest improvement (Dunning & Holmes, 2014; Gibson, Gondoli, Johnson, & Robison, 2014; Sprenger et al., 2013). As noted elsewhere (Logie, 2012), high- and low-ability individuals performing the same complex task may strategically use their cognitive resources differently so that repeatedly practicing the same task over sessions may not address the same cognitive processes to the same degree across all people. Whether the correct explanation is in terms of changes in motivation or strategy, it seems quite possible that improvements on working memory tasks following training do not reflect genuine increases in working memory capacity. It is also possible that some positive effects in this literature reflect the effects of publication bias (because positive effects are more likely to be published than null results).

Conclusions

Our meta-analysis included an impressive number of studies from the burgeoning working memory training literature, but the pattern of results is consistent with an earlier review (Melby-Lervåg & Hulme, 2013). Although there is evidence for near transfer to similar verbal and visuospatial working memory tasks, all immediate and delayed comparisons of nonverbal ability, verbal ability, reading comprehension, word decoding, and arithmetic were not significantly different from zero when compared against treated controls and eliminating outlier studies. Using mediation analyses, we showed that the size of working memory gains was not related to the size of gains on measures of “far transfer.” We conclude that there is no evidence that working memory training yields improvements in so-called far-transfer abilities.

It cannot be concluded from the current review that working memory training could never produce improvements on measures of intelligence or other real-world cognitive skills. However, the extensive efforts in this field to date are discouraging. We believe that current evidence suggests that further attempts to increase working memory capacity by repetitively practicing simple memory tasks on a computer are unlikely to lead to generalized cognitive benefits. We believe new training approaches, likely based on deeper theoretical analyses, will need to be developed and tested if the field of working memory training is to move forward. As we have discussed elsewhere (Redick, Shipstead, Wiemers, Melby-Lervåg, & Hulme, 2015), given the repeated finding that training produces near transfer, training specific skills with interventions that are similar to the targeted outcome will likely be a more fruitful approach than current working memory training programs. In this vein, there is good evidence that difficulties with word reading and problems with reading and language comprehension can be improved by intensive, targeted educational interventions (see Hulme & Melby-Lervåg, 2015). We believe that attempts to produce lasting improvements in attainment and intelligence may be better pursuing these more “conventional” approaches (particularly approaches that involve more varied and stimulating educational interventions) than using repetitive computerized memory games. Finally, we have highlighted a number of critical methodological weaknesses in many studies in this area and have made some recommendations that we believe are important for guiding future studies in the field of cognitive training more generally.