Working memory training in children with borderline intellectual functioning and neuropsychiatric disorders: a triple-blind randomised controlled trial.

BACKGROUND: Poor working memory, lower IQ and maladaptive behaviour form a triple disability known to have negative effects on the academic and social development of children with borderline intellectual functioning (BIF; IQ: 70 < IQ < 85) and neuropsychiatric disorders [attention-deficit hyperactivity disorder (ADHD) and/or autism spectrum disorder (ASD)]. Treatment possibilities for these children are scarce and hardly evidence based. This study primarily investigated whether adaptive computerised working memory training (WMT) may lead to significantly more improvement on a non-trained visuospatial WM task compared with a non-adaptive control WMT (placebo) in children with BIF and neuropsychiatric disorders. As secondary outcome measures, we used the scores on several non-trained neuropsychological near-transfer and far-transfer tasks as well as behavioural measures.
METHOD: We conducted a triple-blind placebo-controlled randomised clinical trial in 72 children (aged 10;0-13;11 years, 53 boys, 19 girls) with BIF and comorbid neuropsychiatric disorders (ADHD = 37, ASD = 21, both = 14) that were referred to child and adolescent psychiatry care, between May 2012 and March 2019. Children completed the Dutch version of Cogmed WMT, either the adaptive training version or the non-adaptive placebo version, 25 sessions (30-45 min a day), for 5 weeks. The primary outcome measure was the score on a non-trained visuospatial working memory task. The primary outcome was measured before and directly after 5 weeks of WMT and again 6 months after training.
RESULTS: A total of 375 children were screened for eligibility and 72 were randomised. No significantly higher levels of improvement over time were found on our primary outcome measure in the experimental WMT group compared with the placebo control WMT, nor in the secondary (near-transfer and far-transfer tasks) or tertiary (behavioural measures) outcome measures. However, this study did show changes over time for these measurements for both the experimental and placebo conditions.
CONCLUSIONS: This study was unable to document superior training effects over time of an adaptive WMT in children with BIF and neuropsychiatric disorders, compared with a placebo (non-adaptive) WMT. The objectively documented changes over time in the non-adaptive WMT arm suggest that these children with persistent impairments in WM may benefit from a structured learning environment that is associated with improvement of neurocognitive functioning and coping strategies. Further research is needed to examine which elements of cognitive training may be useful for which specific patients and to study long-term effects of training.

Children with borderline intellectual functioning (BIF: 70 < IQ < 85) show deficits in neurocognitive functioning and adaptive behaviour. The prevalence of BIF is estimated to be up to 10% (Roeleveld et al. 1997; Simonoff et al. 2006; Westerinen et al. 2017). About one‐third of the children with BIF may have a comorbid neuropsychiatric disorder, the most common being attention‐deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD; Strømme & Diseth 2000). These comorbid disorders may aggravate the problems in adaptive functioning and hinder development. Regular treatments, such as cognitive‐behavioural therapies, are often too complex for children with BIF and ADHD and/or ASD due to lower intellectual abilities and less well‐developed adaptive skills. Furthermore, this group is known to have a lower treatment response to ADHD medication (40–54% responders) compared with patients with an average IQ (70% responders) and also suffer from more serious adverse effects (e.g. Simonoff et al. 2013). Also, these children are more likely to grow up in lower socio‐economic environments and face adversity, which are factors associated with poorer mental health in the general population (Emerson & Brigham 2015). Thus, there is a need to extend the evidence base for effective interventions for these patients. Interventions using computer technology like computerised cognitive training might fill this gap.

Children with BIF are found to have a primary cognitive deficit in working memory (WM; Roording‐Ragetlie et al. 2018). Similarly, ADHD and ASD are also characterised by persistent impairments in WM (Martinussen et al. 2005; Kenworthy et al. 2008; Kasper et al. 2012). Such WM impairments in children with BIF and neuropsychiatric disorders are related to poor adaptive behaviour (Gilotty et al. 2002), heightened vulnerability to academic and social impairments (Cornish et al. 2012) and reduced quality of life (Chiang & Wineman 2014). Therefore, it is important to investigate whether WM can be strengthened in these children. Even a small amount of progress in this cognitive capacity may lead to significant progress in classroom and daily life functioning (Minear & Shah 2006). Furthermore, a computerised working memory training (WMT) might be ideal for children with BIF, because it can take place at home and/or school, appeals to their relatively stronger visual abilities (Van der Molen et al. 2014) and is motivational due to the gaming elements (Dovis et al. 2012; Sadeghi et al. 2020).

Note that several meta‐analyses are critical about the efficacy of WMT. Inconsistent findings within and between studies raise doubts about the long‐term effects and generalisation of the trained task effects (Shipstead et al. 2012; Rapport et al. 2013; Melby‐Lervag et al. 2016). Further, despite improving WM performance, cognitive training had limited effects on ADHD symptoms according to assessments based on blinded measures (Cortese et al. 2015). However, results on the efficacy of computerised WMT in large and diverse patient groups cannot simply be extrapolated to children with BIF and neuropsychiatric disorders due to their unique neurocognitive profile (Danielsson et al. 2012). A study in children with ASD and comorbid ADHD, partially similar to the group in the present study, showed improvement in attention and focus, impulsivity, emotional reactivity and academic achievement after computerised WMT. It has been suggested that computer‐based interventions seem to engage the unique learning style of this population (Weckstein et al. 2017). Furthermore, studies in children with BIF show improvements in short‐term memory (STM) and WM as well as in academic achievements after WMT (Van der Molen et al. 2010; Söderqvist et al. 2012; Danielsson et al. 2015). Moreover, it is known that improvement in training progress varied largely between individual children with intellectual disabilities, due to variability in demographic characteristics (Söderqvist et al. 2012).

Klingberg et al. (2005) studied the difference between an active (adaptive) Cogmed WMT and a placebo (non‐adaptive) version of the training and found a significant improvement on a non‐trained visuospatial WM task in children with ADHD who completed the active WMT. However, that study did not include children with comorbid BIF. Therefore, the main objective in the present study is to examine whether performance on a non‐trained visuospatial WM task shows greater improvement from baseline to endpoint for the adaptive versus the non‐adaptive version of a computerised WMT in children with BIF and neuropsychiatric disorders (ADHD and/or ASD). In the adaptive version, the difficulty level is automatically adjusted, on a trial‐by‐trial basis, to match the WM span of the child on each task. Non‐adaptive means that the difficulty level of the tasks never exceeded the starting level of three items.

Methods

Study design

Children with BIF and neuropsychiatric disorders (ADHD and/or ASD) were recruited from an outpatient facility for paediatric psychiatry in the Netherlands to participate in this triple‐blind controlled clinical trial (between May 2012 and March 2019). The study was characterised as triple‐blind, because the participants (children, parents and teachers), the training coaches and the investigators were all blind to both the treatment condition and the coaching or training progress. Care providers were asked to inform eligible children and their legal representatives about the study, and for written consent on sharing contact details with the research team. A member of the research team then contacted the legal representatives providing them with more information about the study and answering questions. By agreement to participate, representatives were asked for written informed consent, and children provided oral consent (or written consent ≥ 12 years).

After inclusion and exclusion criteria were checked (see section Study Population), children were individually randomised into two groups to evaluate the efficacy of an adaptive WMT (Cogmed version R/M) compared with a non‐adaptive placebo WMT (Placebo version R/M). This study is approved by the Medical Research and Ethics Committee (MREC) region Arnhem‐Nijmegen in the Netherlands, registered under NL32435.091.10. This trial is registered in the Dutch Trial Register, number NL2798 (https://www.trialregister.nl/trial/2798).

The study consisted of four phases (see Fig. 1 for the study flow chart). In the first phase (baseline; T0), the children underwent a neuropsychological assessment, and parents and teachers filled out questionnaires about the child's behavioural symptoms. Second, children performed the training either at home or at school. Third, approximately 1 week after completion of the training (post‐treatment; T1), the neuropsychological assessment was repeated. Parents and teachers again filled out the same set of questionnaires as at T0. Finally, part of the neuropsychological assessment and the same questionnaires were administered once more approximately 6 months after completion of the training (follow‐up; T2).

Figure 1

Study flow chart. ADHD, attention‐deficit hyperactivity disorder; ASD, autism spectrum disorder. [Colour figure can be viewed at wileyonlinelibrary.com]

Study population

Participants were selected by means of four inclusion criteria. Subjects (1) were aged between 10 years 0 months and 13 years 11 months (M = 11.7, SD = 1.2), (2) had a recent IQ score (<1.5 years old) between 70 and 85 (BIF; M = 76.5, SD = 4.8), (3) were classified with a neuropsychiatric disorder by a certified mental health psychologist and/or psychiatrist, that is, ADHD (53%), ASD (29%) or both (18%) according to the DSM‐IV (American Psychiatric Association 2000), and (4) had access to a computer with internet connection and speakers at home and/or at school. Children on medication were only included if there was ‘room for improvement’ with respect to the experienced ADHD symptoms. This was based on a (sub)clinical score on ADHD symptoms according to the DSM‐IV, and stable medication dosages during study participation (ADHD medication n = 29, antipsychotic n = 7 and other n = 4). Based on these criteria, a mental health psychologist determined one's eligibility for participation using information extracted from the electronic medical record, additional ADHD and/or ASD DSM‐IV rating scales (see paragraph Study Outcomes) and a shortened intelligence test if the results of a prior test were >1.5 years old (Kort et al. 2005).

Study interventions

Condition 1: Cogmed Working Memory Training

Children completed the Dutch version of the adaptive Cogmed Working Memory Training (WMT; version R/M; Klingberg et al. 2002). The computerised WMT consisted of 13 verbal and visual STM and WM tasks (Cogmed Cognitive Medical Systems AB Stockholm, Sweden).

Participating children completed eight different tasks during each training session. An example of a verbal WM task is Decoder. In this particular task, letters are read out and lights flash at the same time. When solicited, the child must recall the letter associated with a given light. The program was provided online and used by the child on a personal computer at home and/or school, supervised by a parent and/or teacher. Responses were made by clicking on displays using the computer mouse. The difficulty level was automatically adjusted, on a trial‐by‐trial basis, to match the WM span of the child on each task. Children were assigned a unique ID code and task performance was uploaded in a log file.

During the training sessions, the child received positive verbal feedback from the computer. In addition to this, all‐time high scores were displayed after each task and there was an ‘energy’ counter that could be used on a fun racing game completed after each training day. The racing game was voluntarily and only included as a reward and did not load on WM. After each week of training, the child received a small reward (e.g. choosing what to eat for dinner). Children trained for a period of 5 weeks, 5 days a week (25 days and 200 exercises in total), with an estimated time spent per day between 35 and 45 min. Normally, in clinical practice, a licensed Cogmed coach provides personalised coaching and feedback on the child's performance by telephone, according to a strict protocol (on a weekly basis) with the parent or aide and also with the participating child. Because of the triple‐blind design of our study, the coaching was not personalised but consisted of generic encouragement of the child according to a standardised protocol.

Condition 2: Placebo Cogmed Working Memory Training

The placebo condition was identical to the treatment condition, except for the adaptivity. In the placebo condition, the difficulty level of the tasks never exceeded the starting level of three items. This way, the placebo condition relied less on WM capacity. The placebo WMT works under this assumption that children as young as 4 years old are able to remember at least three chunks of information (Gathercole et al. 2004). By keeping the difficulty level of the trials in the non‐adaptive (placebo) WMT on a low level of one to three items instead of adapting it to match the WM span of the child, this version of the training should not improve one's WM span (Klingberg et al. 2005).

Study outcomes

The primary outcome measure is the score on the Spatial Span task. This task was administered as implemented in the Automated Working Memory Assessment (AWMA; Alloway 2007). The child views a picture of two arbitrary shapes, where the shape on the right has a red dot on it. The child identifies whether the shape on the right is the same as or opposite to the shape on the left. The shape with the red dot may also be rotated. At the end of each trial, the child has to recall the location of each red dot on the shape in sequence, by pointing to a picture with three compass points. There are six items per trial, starting with one stimulus card up to a maximum of six stimulus cards, depending on the performance of the child. If the child answers at least four out of six items correctly (i.e. he or she correctly identifies the sequence of the locations of the red dots), the subsequent trial with an additional stimulus card added to the sequence is performed. Scores are given for every correct answer and can vary from 0 to 36 (6 items 6 trials).

For secondary outcome measures, we used the scores on several near‐transfer tasks (non‐trained verbal and visuospatial STM and verbal WM) and far‐transfer tasks (arithmetic, reading, daily memory, fluid IQ, inhibition control and sustained attention). As tertiary outcomes, we examined six behavioural measures. All secondary and tertiary outcome measures are presented in Table 1.

Table 1

Secondary and tertiary outcome measures

	Secondary outcome measures		Tertiary outcome measures
	Near‐transfer measures	Far‐transfer measures	Behavioural measures
1	Block Recall task (visuospatial STM; Pickering & Gathercole 2001). A researcher taps a sequence of up to nine identical spatially separated blocks, after which the participant is asked to mimic the tapping of that sequence. The amount of correctly mimicked sequences serves as the outcome measure, ranging from 0 to 54.	Sustained Attention Dots reaction time and errors [Amsterdamse Neuropsychologische Taken (ANT); de Sonneville 2009]. Participants should discriminate between patterns consisting of 3, 4 (target signal) or 5 dots. The probability of a yes vs. no response is 1:2. As a consequence, with time‐on‐task a response bias (for the no‐response key) is induced.	Total DSM‐IV symptom scores for ADHD, determined by means of the ADHD DSM‐IV rating scale, as rated by the investigator (DuPaul et al. 1998) using a 4‐point Likert scale (0 = never occurs, 1 = occurs sometimes, 2 = occurs often, 3 = occurs very often). Eighteen symptoms are rated, resulting in a possible range for this measure of 0 to 54.
2	Visual Patterns Test (visual–spatial STM; Della Sala et al. 1997). The participant is shown matrix patterns of black and white squares in grids of different sizes and of increasing difficulty and are asked to memorise these patterns. The amount of correctly remembered patterns serves as the outcome measure, ranging from 0 to 42.	Go–NoGo reaction time and errors (inhibitory control, ANT; de Sonneville 2009). It contains a presentation of Go and NoGo signals in the centre of the screen to indicate whether or not the child should press a button.	Total DSM‐IV symptom scores for ASD, determined by means of the ASD DSM‐IV rating scale specifically made for this study, as rated by the investigator (Supporting Information, Appendix A), using a 4‐point Likert scale (0 = never occurs, 1 = occurs sometimes, 2 = occurs often, 3 = occurs very often). Twelve symptoms are rated, resulting in a possible range for this measure of 0 to 36.
3	Digit Recall task (verbal STM; Pickering & Gathercole 2001). A sequence of up to 8 digits is presented verbally to the participants and they are asked to recall this sequence in forwards order. The outcome measure is the amount of correctly recalled sequences, ranging from 0 to 42.	Raven Standard Progressive Matrices task (fluid IQ; Raven et al. 1996) is made up of five sets of diagrammatic puzzles exhibiting serial change in two dimensions simultaneously. Each puzzle has a part missing, which the participant has to identify among the options provided. The amount of correctly identified missing puzzle parts serves as the outcome measure, ranging from 0 to 60.	The total score on the ADHD‐Vragenlijst (AVL; Scholte & Van der Ploeg 2005), as filled out by parents and teachers. This is an additional measure to determine behavioural symptoms of ADHD. It contains 18 behaviours that are rated by the child's parents in terms of frequency of occurrence in the last 6 months on a 5‐point Likert scale (0 = never, 1 = sometimes, 2 = average, 3 = often, 4 = very often), resulting in a possible total score range from 0 to 72.
4	Non‐Word List Recall task (verbal STM; Pickering & Gathercole 2001). The participant is asked to repeat a series of non‐words of differing syllable length and complexity of sound combinations. The amount of correctly repeated non‐words serves as the outcome measure, ranging from 0 to 36.	Story Recall task (daily memory; Van der Molen 2007). The participant is asked to recall details of a story that is read to him or her. The amount of details correctly recalled serves as the outcome measure, ranging from 0 to 58.	The total score on the Behaviour Rating Inventory of Executive Functioning (BRIEF; Smidts & Huizinga 2009). There are two separate versions of the questionnaire to be completed by the parents or the teacher, to gain insight into the child's executive function problems (experienced daily) at home and at school. Both versions contain 75 behaviours that are rated in terms of frequency of occurrence in the last 6 months on a 3‐point Likert scale (1 = never, 2 = sometimes, 3 = often), resulting in a possible total score between 75 and 225.
5	Backward Digit Recall (verbal WM; Pickering & Gathercole 2001). A sequence of up to 7 digits is presented verbally to the participants and they are asked to recall this sequence in reverse order. The outcome measure is the amount of correctly recalled sequences, ranging from 0 to 36.	A Dutch arithmetic task named ‘Arithmetic Speed test’ (de Vos 1992) assesses arithmetic automation as a measure of academic achievement. Participants are asked five times (add, subtract, multiply, divide, mixed) to correctly answer as many as possible and up to 200 written out automation of mathematics questions of increasing difficulty within 1 min. The amount of total correctly answered questions serves as the outcome measure, ranging from 0 to 200.
6	Listening Recall task (verbal WM; Pickering & Gathercole 2001). Participants are asked to judge the veracity of a series of short sentences and to recall the last word of that sentence (or sequence of last words of sentences in case of multiple items). Outcome measure is the amount of correctly recalled sequences of last words, ranging from 0 to 36.	A Dutch reading task named ‘Reading Speed test’ (Brus & Voeten 1999) measures reading automation as a measure of academic achievement. Children are asked to read as many as possible words of increasing difficulty out loud within 1 min. The amount of words correctly read out serves as the outcome measure, ranging from 0 to 116.

STM, short‐term memory; WM, working memory.

Randomisation

Individual randomisation was performed by an independent person not involved in this research project and provided through sealed envelopes. Four strata were constructed based on sex and diagnosis, to balance the treatment groups with respect to these important characteristics (the stratification factors). A block randomisation schedule with varying block sizes was performed separately within each stratum to reduce the possibility of selection bias. Participants were informed about the allocated intervention after the post‐treatment assessment.

Sample size

Data from the Klingberg et al. (2005) study was used to determine the sample size. They found a mean improvement on their span‐board task of 0.82 (SD 1.01) in the high‐intensity group and 0.15 (SD 0.81) in the low‐intensity group after WMT. An a priori sample size calculation suggested that with an alpha error level of 0.5% and an expected drop‐out rate of 10%, a total sample size of 100 would give a power of 81.2% to detect between‐group treatment effects with an effect size of 0.6 (Klingberg et al. 2005).

Statistical analysis

All statistical analyses were conducted using IBM SPSS Statistics for Windows, version 24.0.0 (IBM Corp. 2016). First, we checked if participants in the experimental group and placebo group were sufficiently comparable in terms of descriptive statistics.

Our primary analysis was conducted using an intent‐to‐treat approach and therefore included all randomised patients who were willing to complete, both, a pre‐WMT and post‐WMT neuropsychological assessment. A two‐way repeated measures analysis of variance (ANOVA) was used to determine whether any change on the Spatial Span task (i.e. the dependent variable) is the result of the interaction between the between‐subjects factor Treatment condition (Experimental/Placebo) and the within‐subjects factor Time. A similar repeated measures ANOVA was conducted for each of the secondary (n = 14) and tertiary (n = 6) outcomes. Each model contained the same within‐subjects factor (Time) and a between‐subjects factor (Treatment condition; Experimental/Placebo). The repeated measures ANOVA approach was deemed most suitable to address the study's objective of comparing the levels of improvement in the two treatment conditions rather than comparing the post‐WMT scores in the two conditions while adjusting for baseline scores.

For the behavioural measures of ADHD [ADHD rating scale, ADHD‐Vragenlijst (AVL) parents and AVL teacher], only participants with a diagnosis of ADHD or ASD with comorbid ADHD were included in the analyses (n = 51) and not the participants with ASD only. For the behavioural measure of ASD (ASD rating scale), only participants with a diagnosis of ASD and ASD with comorbid ADHD were included in the analysis (n = 35) and not the participants with ADHD only. For the remaining measures, all participants were included in the analyses.

Statistical assumptions inherent to the repeated measures ANOVA were checked for each dependent variable. The scores for Listening Recall, Visual Patterns Test, reaction time of Sustained Attention Dots and the error scores of both Sustained Attention Dots and Go–NoGo did not adhere to the assumption of normality. Applying a log transformation to these variables adjusted their distribution sufficiently for all but the reaction time for Sustained Attention Dots and the error score of Go–NoGo. Analyses for the Listening Recall score, Visual Patterns Test score and Sustained Attention Dots error score were performed with the log‐transformed version of the variable to retain as much statistical power as possible.

A total of 15 outliers (>3 SD from the mean for the respective time point) were identified across all tests and time points. They were retained in the dataset due to insufficient clinical reason to remove them. In total, 4% of the data were missing. Little's Missing Completely At Random (MCAR) test showed that no specific patterns could be identified for the primary outcome measure (Spatial Span; χ 2 = 2.367, df = 2, P = 0.306) and list‐wise deletion was applied across all analyses. The significance level was set at the threshold of P = 0.050, and a Bonferroni correction was performed to evaluate the impact of multiple comparisons on the same sample for the neurocognitive as well as for the AVL and Behaviour Rating Inventory of Executive Functioning (BRIEF) scores (n = 19, P = 0.003). Post hoc pairwise comparisons were conducted with a Bonferroni correction as well.

Results

A total of 375 children were screened for eligibility and 295 met eligibility criteria (Fig. 1). Although eligible, candidates often refused to participate (66% of screened and eligible candidates). Other reasons for not enrolling are described in Fig. 1. Finally, a total of 76 children enrolled in the study. Four dropped out before completing a minimum of 60% of the training (after 0, 6, 10 and 13 sessions out of 25) and refused to participate in post‐measurements. These participants were excluded, resulting in a total sample size of 72. Eight out of these participants also did not complete all 25 sessions because of motivational problems but did reach the minimum of 60% (between 15 and 24 sessions). The fidelity of the coaching assessed was 100%. All participants received the agreed five coaching sessions (once a week) by the licensed Cogmed coach by telephone. Attrition rate was low (4%) and did not significantly differ between the experimental and placebo conditions. In total, 97% of the participants in the experimental condition and 95% of the participants in the placebo condition completed all three measurements. Controlling for the initial AVL return rate of 68% for parents and 63% for teachers at T0, we achieved a subsequent 98% and 92% return rate for parents at T1 and T2, respectively, and a 100% and 89% return rate for teachers. Controlling for the initial BRIEF return rate of 97% for parents and 93% for teachers at T0, we achieved a subsequent 96% and 93% return rate for parents at T1 and T2, respectively, and a 93% and 82% return rate for teachers.

Table 2 shows descriptive statistics of the experimental and placebo conditions. While the two treatment conditions were mostly comparable, mean treatment duration in the experimental condition (M = 714.3, SD = 199.7) was longer than in the placebo condition (M = 564.7, SD = 99.3), a statistically significant difference of M = 149.6 min, t(69) = 4.05, P ≤ 0.001. Subsequent Pearson correlations showed that treatment duration was not significantly related to any of the dependent variables, indicating limited necessity to control for the difference in training duration in the subsequent main analyses. Descriptive statistics for all outcome measures per condition and per time point can be found in Supporting Information, Appendix B.

Table 2

Descriptive statistics of IQ, age, treatment duration, sex and diagnosis per treatment condition

		Experimental condition (N = 35)	Placebo condition (N = 37)	P‐value †
		M ± SD	M ± SD
IQ		76.63 ± 4.74	76.86 ± 4.96	0.837
Age (in years)		11.69 ± 1.32	11.76 ± 1.09	0.804
Treatment duration (min)		714.32 ± 199.65	564.70 ± 99.31	<0.001

Independent samples t‐test.

Chi‐square test.

ADHD, attention‐deficit hyperactivity disorder; ASD, autism spectrum disorder.

Visuospatial working memory

A significant time effect (P < 0.003) was found on the score of the Spatial Span task at T1. As is illustrated in Fig. 2, the scores significantly improved over time and the improvement was similar for both the experimental and placebo conditions. No other main and interaction effects were found. All model statistics can be found in Supporting Information, Appendix C, and post hoc comparisons of the significant time effects can be found in Supporting Information, Appendix D.

Figure 2

Line graphs for the primary outcome measure of visuospatial working memory. Note. * Measure for which a significant main effect of time (P < 0.003) was found.

Near‐transfer and far‐transfer tasks

A significant time effect (P < 0.003) was found on the scores of Block Recall, Digit Recall, Non‐Word List, Backwards Digit Recall, the Raven Standard Progressive Matrices, Story Recall, Arithmetic Speed test, Reading Speed test and both the reaction times of Sustained Attention Dots and the Go‐No Go task. As illustrated in Figs 3 and 4, these measures significantly improved over time. No other main and interaction effects were found for any of the near‐transfer and far‐transfer measures. All model statistics can be found in Supporting Information, Appendix C, and post hoc comparisons of the significant time effects can be found in Supporting Information, Appendix D.

Figure 3

Line graphs for the secondary near‐transfer outcome measures. Note. * Measure for which a significant main effect of time (P < 0.003) was found.

Figure 4

Line graphs for the secondary far‐transfer outcome measures. Note. * Measure for which a significant main effect of time (P < 0.003) was found.

Behavioural measures

A significant time effect was found on the ADHD ratings by the investigator (P < 0.05), the ASD ratings by the investigator (P < 0.05), the AVL filled out by parents (P < 0.003) and the BRIEF filled out by parents (P < 0.003). As is illustrated in Fig. 5, these measures significantly improved over time and these improvements were similar for both the experimental and placebo conditions. No other main and interaction effects were found for the behavioural measures. All model statistics can be found in Supporting Information, Appendix C, and post hoc comparisons of the significant time effects can be found in Supporting Information, Appendix D.

Figure 5

Line graphs for the tertiary behavioural outcome measures. Note. * Measure for which a significant main effect of time (P < 0.05 for ADHD and ASD rating scales; P < 0.003 for AVL and BRIEF) was found. ADHD, attention‐deficit hyperactivity disorder; ASD, autism spectrum disorder; AVL, ADHD‐Vragenlijst (ADHD questionnaire); BRIEF, Behaviour Rating Inventory of Executive Functioning.

Discussion

In contrast to our hypothesis, the results of this triple‐blind randomised placebo‐controlled study in children with BIF and neuropsychiatric disorders did not show significant larger improvement over time on our primary outcome measure (visuospatial WM) in the experimental WMT group compared with the placebo control WMT. Furthermore, no additional effects in favour of the experimental WMT were found on any of the secondary (near‐transfer and far‐transfer tasks) or tertiary (behavioural measures) outcomes. These findings are consistent with the outcome of several meta‐analyses and reviews on Cogmed WMT (Shipstead et al. 2012; Chacko et al. 2013; Melby‐Lervåg & Hulme 2013; Rapport et al. 2013; Sonuga‐Barke et al. 2013; Cortese et al. 2015) and a recent study in youth with ADHD under pharmacological treatment (Dentz et al. 2020).

However, this study did show improvements over time for both the experimental and non‐adaptive control conditions on objectively measured non‐trained visuospatial WM as well as on near‐transfer and far‐transfer neurocognitive measures and on behavioural measures (ADHD and ASD ratings by the investigator, AVL parents and BRIEF parents). Although no significantly higher levels of improvement over time were found in the experimental group compared with the control group, the results may suggest that these vulnerable children seem to be able to improve their WM capacity in a relatively short amount of time with (non‐)adaptive training.

The Cogmed WM improvement index, provided by the training software to quantify compliance and performance in the adaptive version of the training, was 26.20 points (SD = 14.88) in our study. This is much higher than most improvement indices reported in other studies. For example, Berger et al. (2020) reported an improvement index of 20.76 points in children functioning at a lower or average neurocognitive level. Furthermore, they found that training effects did not differ significantly between children functioning at a lower neurocognitive level and those who had an average level (Berger et al. 2020), which seems to be in line with the findings in our study.

An explanation for the lack of difference in effectiveness between the experimental and placebo training groups might be found in our study design. Van der Molen et al. (2010) studied a group of children that was quite similar to our patient group and found significant positive training effects on (non‐trained) neurocognitive measures. However, in this study, a third group, a waiting list control group, was used. Positive effects of training were only seen in the active training group compared with the waiting list control group and not compared with a placebo (non‐adaptive training) control group. The positive results in both groups in our study were not controlled for specific conditional factors common to both the experimental and control groups, which may have resulted in improvements in both groups. Both interventions required continued perseverance, sustained attention, inhibition skills and frustration tolerance and may have trained specific coping skills or even increased the sense of self‐efficacy and self‐esteem. Yet, the positive training results in our study might be of greater significance when compared with a waiting list control group.

A second explanation for the lack of difference in effectiveness between the experimental and placebo training groups might be found in the intensity of WM training, the only difference between the experimental and placebo conditions. The treatment condition (adaptive training) had an automatically adjusted level of difficulty implemented for each task on a trial‐by‐trial basis to match the WM span of the child. Perhaps the experimental condition did not meet the high intensity needed to accomplish significant training effects, despite the adaptive factor. Maybe a higher intensity training, more training minutes a day or a prolonged training period in the experimental condition would lead to significant differences with the placebo condition. In contrast, the difficulty level of the tasks in the placebo condition (non‐adaptive control training) never exceeded the starting level of three items. This way, the placebo condition encompassed a lower WM load (i.e. a low number of items to remember). However, in our study, 8% of the participating children started with a particularly low verbal WM baseline (Backwards Digit Recall and Listening Recall; <3 items) at pre‐treatment and did improve to a score of ≥3 after training was completed, which means that, in this study, some of the children in the placebo condition trained their verbal WM capacity up to a capacity of three. Besides the lower intensity in WM load in the placebo condition, there is also a significant lower intensity in total training minutes a day (Table 2). Both factors might be of positive influence on the participating children's motivation for training. Training is, in general, more difficult for this population. A pilot study indicated that Cogmed WMT protocols containing more training days with shorter training durations per day may lead to similar or even better training effects compared with the standard protocol, even with less total training time (Mawjee et al. 2014). This is consistent with the general treatment approach for children with mild intellectual disabilities. These children may have a shorter attention span and need shorter duration per each session (Dutch Knowledge Centre on MID/Landelijk Kenniscentrum LVB 2012). Therefore, in these cases, the placebo condition may represent a low intensive experimental condition. A third explanation for the lack of significant difference between the two training groups may be a lack of active coaching in the experimental condition. Due to the triple‐blind design, coaching was limited to generic encouragement and motivation, and – in contrast to Cogmed in clinical practice – coaching could not be based on individual training results because the coach was blinded to group assignment. The active weekly coaching is found to be one of the most important components in a training program with the aim of improving ADHD symptoms (Sonuga‐Barke et al. 2013). Future studies should pay more attention to non‐specific therapeutic effects and may consider coaching based on real training results and an active role for the (unblended) coach (e.g. weekly face‐to‐face contacts).

Despite the absence of significant superior trainings effects, our findings may suggest that this particular patient group may benefit from WMT, whether adaptive or non‐adaptive, because the WM capacity of all children improved. Improvement of WM capacity is found to be associated with changes in the dopaminergic system (McNab et al. 2009). These changes enable more efficient general information processing in daily life because WM and attention and other executive functions are closely related (McNab et al. 2009). This association on neurobiological level illustrates how WM capacity, attention, daily functioning and academic achievement are linked. Furthermore, the cognitive training provides a structured learning environment in which neurocognitive functioning may ameliorate and children may pick up coping strategies such as perseverance and frustration tolerance. This may increase the sense of self‐efficacy and self‐esteem.

This study had some limitations. As noted before, our sample size was limited due to a large number of refusals to participate. A selection bias could therefore not be ruled out. Second, a waiting list control group was lacking, with the result that we could not control for specific conditional factors common to both groups. Third, the training was not embedded in daily life. Diamond & Lee (2011) demonstrated that isolated neurocognitive training may not be adequate to improve executive functioning in daily life. Further research is recommended to find out if an extended WMT including individualised coaching and embedded in daily life may be an effective intervention for children with BIF and neuropsychiatric disorders.

Source of Funding

This project is funded by MIND Netherlands and the Dutch WKK foundation. They did not have any role during execution, analyses, interpretation of the data or decision to submit results.

Conflict of Interest

The authors report no conflicts of interest.

Registration

We adhered to CONSORT guidelines. The Medical Research and Ethics Committee (MREC) of Radboud University in the Netherlands has approved this study, registered under NL32435.091.10.

Trial registration database: Dutch Trial Register, register number NL2798.

Data S1. Supporting information

Click here for additional data file.