The modality and redundancy effects in multimedia learning in children with dyslexia.

The present study aimed to examine the modality and redundancy effects in multimedia learning in children with dyslexia in order to find out whether their learning benefits from written and/or spoken text with pictures. We compared study time and knowledge gain in 26 11-year-old children with dyslexia and 38 typically reading peers in a within-subjects design. All children were presented with a series of user-paced multimedia lessons in 3 conditions: pictorial information presented with (a) written text, (b) audio, or (c) combined text and audio. We also examined whether children's learning outcomes were related to their working memory. With respect to study time, we found modality and reversed redundancy effects. Children with dyslexia spent more time learning in the text condition, compared with the audio condition and the combined text-and-audio condition. Regarding knowledge gain, no modality or redundancy effects were evidenced. Although the groups differed on working memory, it did not influence the modality or redundancy effect on study time or knowledge gain. In multimedia learning, it thus is more efficient to provide children with dyslexia with audio or with auditory support.

For efficient user‐paced multimedia learning, provide children with dyslexia information in an auditory way or with auditory support.

INTRODUCTION

There is continuous debate on how children with dyslexia can be best supported in their learning. Due to a phonological deficit and accompanying working memory problems, children with dyslexia have problems with learning from text (e.g., Berninger, Raskind, Richards, Abbott, & Stock, 2008; Swanson, Zheng, & Jerman, 2009). Multimedia may support their learning by replacing written text with audio or by adding audio to the written text. According to principles put forward in the cognitive theory of multimedia learning (Mayer, 2005), various types of multimedia may impact children's learning. The modality effect entails a larger learning effect for spoken text with pictures than for written texts with pictures (Mayer, 2005). However, this effect tends to reverse over time (e.g., Savoji, Hassanabadi, & Fasihipour, 2011; Scheiter, Schüler, Gerjets, Huk, & Hesse, 2014; Tabbers, Martens, & Merriënboer, 2004). Furthermore, many studies have failed to replicate such a modality effect, which can be attributed to boundary conditions, such as pacing of the learning material (Tabbers, 2002). Evidence for a so‐called redundancy effect has also been found in that presenting identical information in different modalities simultaneously may hamper the learning process (i.e., Gerjets, Scheiter, Opfermann, Hesse, & Eysink, 2009; Mayer, 2005; Mayer, Heiser, & Lonn, 2001). It is by no means clear how the modality effect and the redundancy effect apply to children with dyslexia. Therefore, in the present research, we examined whether these effects would affect the efficiency and knowledge gain in multimedia learning in children with dyslexia to the same extent as their typically reading peers while taking into account children's working memory capacity.

Modality effect in multimedia learning

Information in learning situations is increasingly provided in multimedia form: input of both words (written/spoken) and visualizations (pictures/animations; Mayer, 2005). The dual‐channel theory describes how sensory information is processed through both an auditory channel and a visual channel in working memory, which are seen as parallel and equal (Baddeley, 1995). Together with the limited‐capacity theory (Baddeley, 1995), assuming that working memory can only process a certain amount of information at a time, it forms the basis of the cognitive theory of multimedia learning (CTML, Mayer, 2005). According to the CTML, it can be assumed that both the auditory and visual channels have a maximum capacity and that more information processing is possible when the two channels are combined. It is claimed that spoken texts with pictures have a larger learning effect than written texts with pictures (i.e., the modality effect, Mayer, 2005) because of the fact that the combination effectively triggers both the auditory and visual channels with less risk of information overload as is the case with written text with pictures. The CTML states that both recall of facts (retention) and applying learned information to a new situation (transfer) is better when the material is presented as spoken texts with pictures instead of written text with pictures. Most research has been done in system‐paced environments, in which the software, and not the user, is in control of the study time. A meta‐analysis of Ginns (2005), including 43 studies on the modality effect representing the performance of 1,887 students, indeed showed that people learn more from spoken text with pictures than from written text with pictures, with a moderate to large effect size.

Although the modality effect thus has been evidenced directly after learning, many studies have failed to replicate the modality effect. This may be due to the fact that in all of the above studies, multimedia were presented in a system‐paced learning environment. Studies investigating the effect in a user‐paced system or over time showed no or even reversed modality effects (Savoji et al., 2011; Scheiter et al., 2014; Schmidt‐Weigand, Kohnert, & Glowalla, 2010; Segers, Verhoeven, & Hulstijn‐Hendrikse, 2008; Tabbers et al., 2004; van den Broek, Segers, & Verhoeven, 2014; Witteman & Segers, 2010). In a user‐paced learning environment, students control the speed of the lessons. In contrast to learning in a system‐paced setting, Tabbers, Martens, and Van Merriënboer (2001) found no modality effects on learning gain (retention or transfer knowledge) in a user‐paced learning environment. They also found reversed modality effects on retention or transfer knowledge in such a learning environment (Tabbers et al., 2004). Regarding retention knowledge, Witteman and Segers (2010) also showed a reversed modality effect directly after the lessons. A theoretical explanation for this superior effect of reading could be that reading activates both orthography and phonology and thus creates a double‐memory trace (see Nelson, Balass, & Perfetti, 2005). Thus, in the user‐paced learning environment where children can determine their own pace, they seem to learn as much or even more from written text with pictures than from spoken text with pictures.

It has also been evidenced that modality effects tended to disappear in a user‐paced learning environment on the long term. For example, Segers et al. (2008) showed that in primary school children (11‐year‐olds), learning from written text with pictures in the long term was more effective than learning from spoken text with pictures. The modality effect directly after learning on retention questions disappeared after 1 week. Also, with regard to transfer questions, directly after the lessons, a modality effect was found, whereas a week later, a reversed modality effect could be observed. Witteman and Segers (2010) as well as She and Chen (2009) showed long‐term reversed modality effects for transfer knowledge but no effect on retention. Both sixth graders (Witteman & Segers, 2010) and seventh graders (She & Chen, 2009) learned in the long term more from text with pictures than from audio with pictures. In adults, reversed modality effects were found even after one night, on both retention and transfer knowledge (van den Broek et al., 2014). A direct comparison between both short‐ and long‐term system‐ and user‐paced learning environments was made by Ruf, Seckler, and Opwis (2014). They showed a reversed modality effect over time in both the system‐ and user‐paced settings: Learning from text and pictures led to more learning gain in the long term.

The disappearance of the modality effect in a user‐paced system may be explained by the same theories underlying the modality effect in a system‐paced environment: after all, the limited‐capacity theory (Baddeley, 1995) states that working memory can only process a certain amount of information at a time. So when given enough time, the reader can process both the text and pictures (dual theory: Baddeley, 1995). In reading, the reader can in fact create a better understanding as it is easier to go back and forth in the text (van den Broek et al., 2014). The reversed modality effect thus can be explained by the fact that spoken text is transient, whereas written text remains on the screen and can be absorbed longer. Singh, Marcus, and Ayres (2012) indeed showed this transient information effect: Written text led to a larger learning gain than the (identical) spoken text. They argue that this is due to the extra cognitive load longer spoken texts create because of its lack of permanency.

Redundancy effect in multimedia learning

Presenting identical information in different multimedia forms simultaneously, for example, providing a text on screen and reading that text out loud, is considered to be providing the learner with redundant information. The CTML states that instead of enhancing learning, redundant information hampers the learning process because it requires extra working memory capacity, which is no longer available for learning (Mayer, 2005). Redundant information can be in the form of written text, when an audio condition is compared with a text–audio condition, or in the form of audio, when comparing a written text condition with a text–audio condition. The redundancy effect is very robust, and many studies have shown this effect, especially when the redundant information was in the form of written text (Mayer, 2005). For example, Kalyuga, Chandler, and Sweller (1999) showed that students learned more from a diagram with spoken text than from a diagram with spoken‐and‐written text. Providing students with identical (redundant) information hinders their learning. The redundancy effect was also evidenced in a study by Mayer and Moreno (2002) in which students examined an animation about lightning formation. Half of the participants were also presented with redundant on‐screen text. Results showed that adding the same text to presented narration and animation led to decreased retention and transfer knowledge. Mayer and Johnson (2008) added to this finding that redundant information hindered learning when it consumes cognitive load that is essential for processing the material: when the narrative text is also presented on screen. Also, Jamet and Le Bohec (2007) demonstrated the redundancy effect on both retention and transfer knowledge. Students learned more from diagrams with spoken information, compared with adding the same information as written text to the materials.

Next to redundant information in the form of written text as described above, redundant information can also be in the form of spoken text. Only few studies have looked into this aspect of the redundancy effect. For example, Diao and Sweller (2007) showed redundancy effects on redundant aural information. They examined reading comprehension in second‐language learners and compared reading comprehension between written text and written and spoken text containing the same information. Students could comprehend the information better when only presented with text, compared with simultaneous presenting of written and spoken text. Moreno and Mayer (2002) compared pictures accompanied by written text, spoken text, and written‐and‐spoken text in a virtual‐reality environment. Contrary to Diao and Sweller (2007), they found a reversed redundancy effect on audio information: Text alone led to less retention and transfer knowledge compared with the combination of written and spoken text. The findings were attributed to the virtual‐reality environment: Moreno and Mayer (2002) argued that students were perhaps more inclined to look around and observe instead of reading the textual material. Likewise, in a hypermedia study, Gerjets et al. (2009) showed that students learned more from written text only than from spoken or combined written and spoken text.

So, in general, the redundancy effect is clear when the redundant information is in written form and added to aurally presented information with pictures. When the redundant information is in oral form, and added to a written text, there are contradicting results on whether written text combined with audio benefits the learning outcomes compared with text only. Moreover, all these studies focused on redundancy effects directly after learning. Studies on long‐term effects of the redundant information are generally lacking.

Multimedia learning in children with dyslexia

Dyslexia is an impairment in reading and spelling, given adequate intelligence and educational opportunities, which is in particular associated with a phonological core deficit (Lyon, Shaywitz, & Shaywitz, 2003). Although children with dyslexia are often provided with (extra) audio to support their reading, little is known on the effects of multimedia on learning in these children. Only a small number of studies have been conducted on multimedia learning in people with dyslexia. As children with dyslexia often have lower working memory—an important aspect in multimedia learning—they may learn differently in a multimedia setting.

Many studies have shown children with dyslexia to be impaired on verbal working memory tasks (e.g., Beneventi, Tønnessen, Ersland, & Hugdahl, 2010; Berninger et al., 2008; Menghini, Finzi, Carlesimo, & Vicari, 2011; Swanson et al., 2009; Tijms, 2004). However, there is debate on whether this relies on their phonological core deficit or not. For example, Smith‐Spark and Fisk (2007) found that when phonological differences are taken into account, children with dyslexia still show working memory deficits, whereas Schuchardt, Maehler, and Hasselhorn (2008) found that differences between children with and without dyslexia disappear when controlling for phonological differences. Pickering (2012) argued that children with dyslexia have difficulties with the phonological aspects of working memory and the central executive function of working memory.

Visual working memory is less common to be measured in people with dyslexia, and there is no consensus whether it is impaired in children with dyslexia. Menghini et al. (2011) showed that working memory in children with dyslexia was impaired in the phonological loop, as well as in visual aspects of the working memory. In a similar vein, Reiter, Tucha, and Lange (2005) showed differences between children with and without dyslexia on visual working memory, just like Smith‐Spark and Fisk (2007) found adults with dyslexia to be impaired on visuospatial working memory. On the other hand, Jeffries and Everatt (2004) found no differences between children with and without dyslexia on visuospatial working memory tasks.

On the basis of the CTML (Mayer, 2005), children with working memory problems, such as children with dyslexia, would benefit more from spoken text with pictures compared with written text with pictures (larger modality effect), as they would be more susceptible to cognitive overload in the written text condition. In a similar vein, combining written and spoken text with pictures would also create extra cognitive overload for these children (larger redundancy effect). However, the existing studies examining multimedia learning in dyslexia are contradictive. Audio support (in children with dyslexia) most often focuses on word recognition and phonological skills (e.g., Magnan & Ecalle, 2006; Underwood, 2000), in other words, on reading. Little is known on the effects of multimedia on knowledge learning in children with dyslexia. One of the few studies on multimedia learning in people with dyslexia is from Alty, Al‐Sharrah, and Beacham (2006). They investigated learning from different media combinations in university students with and without dyslexia when studying statistics in an e‐learning environment. Students with dyslexia performed better in a text‐only condition, compared with text and diagrams and with audio and diagrams, whereas typically developing students scored higher in the sound‐and‐diagram condition. This is in contrast to the expectation that text only would hinder the students with dyslexia, and Alty et al. (2006) suggested that this has to do with compensating strategies in the students with dyslexia. It could also be explained by the finding of Harrar et al. (2014) that people with dyslexia have a larger cost when switching their attention from visual information to audio‐presented information, leading to more or faster cognitive (over)load. Beacham and Alty (2006) also showed differences between students with and without dyslexia with respect to different multimedia learning environments; however, they did not find one specific media condition that is more beneficial for all students with dyslexia. With regard to the redundancy effect, Lallier, Donnadieu, and Valdois (2013) showed that children with dyslexia have more difficulties processing verbal‐ and audio‐presented information simultaneously. This is in line with the CTML (Mayer, 2005), that children with working memory problems would learn less in a combined multimedia environment and thus would show larger redundancy effects.

Although working memory is theoretically related to the modality and redundancy effects, the relation of working memory to these effects has hardly been examined in typically developing children, let alone in children with dyslexia. Only Witteman and Segers (2010) examined individual differences in working memory in a user‐paced learning environment in typically developing children but found no relation of working memory with the modality effect.

Because children with dyslexia experience reading difficulties and generally read slower than typically developing children (Shaywitz & Shaywitz, 2001), it is important to also take study time into account during their multimedia learning. These children are expected to need more study time when they have to read a text themselves, compared with a situation where information is presented to them aurally. Kim, Lombardino, Cowles, and Altmann (2014) examined college students with developmental dyslexia in an eye‐tracking study on the comprehension of graphs. They found that students with dyslexia processed visual presented information (text and picture) differently compared with their typically developing peers. Students with dyslexia needed more time to process both linguistic (the text) and nonlinguistic (the graphs) stimuli. Study time is thus an important aspect when focusing on efficient learning.

The present study

To sum up, children with dyslexia read slower, which increases their study time, and they may have lower working memory capacities. In practice, they are often provided with multimedia (audio‐only or audio support added to a written text) to compensate their reading problems. Indeed, multimedia offers various possibilities for supporting learning in children with dyslexia; however, it can also hinder their learning due to cognitive overload. Given the discrepancy between what could be expected based on theory and the few contradicting studies on multimedia learning in people with dyslexia, it is by no means clear whether the optimal way of presenting information to typically developing children is also the optimal way of presenting information to children with dyslexia.

In the present study, we aimed to examine the impact of modality and redundancy effects on efficiency and knowledge gains in multimedia learning in children with dyslexia. The research questions were (a) to what extent do modality and redundancy effects have the same impact on the study time and knowledge gain in children with dyslexia as compared with typically reading peers and (b) to what extent are individual differences in children's working memory capacity related to these effects. Consideration was given to both retention and transfer knowledge, on both the short and long terms, in a realistic, user‐paced learning environment.

In order to answer these research questions, children with dyslexia and a control group of typically developing children were presented with three different types of user‐paced multimedia lessons in a within‐subjects design: pictorial information presented with (a) written text, (b) audio, or (c) combined text and audio. Children were tested on retention and transfer questions directly after studying and after 1 week.

With regard to study time, it was hypothesized that in children with dyslexia, compared with their typically developing peers, larger modality and redundancy effects would be observed. Children with dyslexia were expected to spend more time in lessons with written text than in lessons with audio. With respect to knowledge gain, on the basis of the CTML, also stronger modality and redundancy effects in children with dyslexia were expected starting from the assumption that these children would be more susceptible to cognitive overload. In a similar vein, it was expected that poorer working memory would lead to larger modality and redundancy effects. However, in light of the literature reviewed above, one could also expect no or reversed modality effects on knowledge gain in typically developing children due to the user‐paced learning environment. In children with dyslexia, differences could then be expected in favour of the text condition, which would lead to smaller modality effects, due to the transiency of audio.

METHOD

Participants

Out of an existing database of 550 school, 13 schools in the central region of the Netherlands signed up to participate. Informed active consent was obtained from the parents and the schools before children were allowed to participate. This study was approved by the Ethics Committee of the Faculty of Social Sciences of our university.

All children with dyslexia in this research were officially diagnosed with dyslexia and in possession of an official dyslexia statement provided by a certified child psychologist according to the clinical assessment of the Protocol Dyslexia Diagnosis and Treatment. The Protocol Dyslexia Diagnosis and Treatment is a guide to diagnosing, indicating, and treating clients with dyslexia with the aim of describing optimal care for clients with dyslexia based on current scientific, professional, and social insights (Blomert, 2006). The control group was selected from the same classrooms as the children with dyslexia to diminish group influence. In total, 38 typically developing children (22 boys) aged 10.92 years (SD = 0.37) and 26 children with dyslexia (13 boys) aged 11.22 years (SD = 0.53) participated in this research (64 in total). Only monolingual children with no developmental deficits (only dyslexia) were included in the research.

Procedure

Children were tested between January 2016 and April 2016 by five undergraduate students. Before data collection started, they received training twice (each 2.5 hr) on the lessons and tests. Testing was done in an individual setting at a quiet room in school. The children were tested for 45 min/week, 4 weeks in a row. All 64 children were provided with three multimedia lessons offered in a randomized‐block design with lessons, modalities, and posttests randomized per child. So all children studied every lesson once (one lesson a week). During studying, children's learning time was recorded. After the lessons, children immediately filled out the first posttest to measure the learning effect in the short term. They did not receive feedback on their answers. The second, alternative version of the posttest was administered a week later to measure long‐term effects. In addition, some other tests were performed on working memory, non‐verbal reasoning, and language. For five children, not all data were complete, due to absence during one of the measurements.

Measures

General non‐verbal intelligence

Raven's (2006) Progressive Matrices General was used to measure non‐verbal intelligence and administered according to its individual assessment instructions. Sixty visual patterns of increasing difficulty were presented (A–E). In each pattern, children had to choose the missing piece of information from six or eight alternatives. Raw scores (number of correct answers) were used for analysis. In the present study, Cronbach's alpha was .84, indicating good reliability.

Word decoding

The Een‐Minuut‐Test (One‐Minute Test), was used to measure the children's word decoding (Verhoeven, 1995). The Een‐Minuut‐Test is a standardized test that consists of a reading card with different words in increasing difficulty level. Children have to correctly read out loud as many words as possible in 1 min. The number of correct read words in 1 min was used for analysis.

Pseudo‐word decoding

The Klepel was used to measure the children's pseudo‐word decoding (Verhoeven, 1995).The Klepel is a standardized test that consists of a reading card with different pseudo‐words (non‐existing words) in increasing difficulty level. Children have to correctly read out loud as many pseudo‐words as possible in 1 min. The number of correctly read words in 1 min was used for analysis.

Verbal working memory

The subtest digits backwards of the Dutch version Wechsler Intelligence Scale for Children III (Wechsler, 2005) was used to measure verbal working memory and administered according to its individual assessment instructions. Children had to recall a sequence of spoken digits (between two and nine). Children were asked to recall the sequence backwards, for example, when the sequence 5–4–7 was provided, children had to recall 7–4–5. The number of digits in a list increased by one, until two sequences of the same length were incorrect. There were no time limits. The score given was the number of correct recalled lists. Higher scores reflected better performance. Raw scores were converted into standardized values for analyses.

Visual working memory

An N‐backwards working memory task with N = 2 (a variant of the ‘n‐back’ procedure of Gevins & Cutillo, 1993) was used to measure visual working memory. This task is commonly used in literature as a working memory measure (Baddeley, 2003) and useful in experimental research (Jaeggi, Buschkuehl, Perrig, & Meier, 2010). On a laptop screen (1,366 × 768 pixels), children were presented with numbers (one at a time) and had to press a key whenever they saw a number that repeated after two intervening stimuli (N = 2). For example, children saw the sequence 2–5–2 and had to press the key at the second 2. Stimuli were presented for 600 ms with 645 ms in between. Children were presented with 225 stimuli, of which 32 were an N = 2 item. The score given was the number of correct responses. Higher scores reflected better performance. Raw scores were converted into standardized values for analyses.

Multimedia lessons

All children made three multimedia lessons, namely, balance in nature, motion, and global warmth in different types (modalities) of learner‐paced multimedia lessons: pictorial information presented with (a) written text, (b) audio, or (c) combined text and audio. One lesson consisted of 12 slides, including a title page. The children were able to move back and forth through the pages at their own pace. The lessons were based on a text book of Grade 6 (1 year above children's school year; van Hoof, Siemensma, Smit, & Vegh, 2009) to ensure that the children have not had these lessons and to enable the possibility of learning gain. Pictures were also from the same schoolbook or, when unavailable, from the Internet open source. The schoolbook from which the lessons were taken provides a very similar build‐up per lesson. The lessons were thus comparable, and they each involved approximately 530 words.

Pictures are expected to support learning (CTML: Mayer, 2005). There are five kinds of pictures, which decrease in added value for learning (Carney & Levin, 2002): transformational, interpretational, organizational, representational, and decorative. The first four categories are considered to be beneficial for learning, whereas decorative pictures are not. To determine the relevance of the pictures used in this study, 11 educational experts (PhD students in Educational Science) judged the pictures and labelled them in the five categories. Of all the pictures in this study, 6.57% was labelled as transformational, 10.35% as interpretational, 10.61% as organizational, 35.61% as representational, and 36.87% as decorative pictures. So, almost two thirds of the pictures can be considered to be beneficial for learning. The used pictures were a good reflection of pictures used in schoolbooks, thus adding to the realistic learning environment we aimed to replicate.

Knowledge gain

Children studied every lesson once, with two posttests (directly after learning and 1 week later). The posttests consisted of both retention and transfer questions. Children were presented with eight retention and four transfer questions per test. The retention questions were multiple‐choice questions, for example, “The vertebral column provides protection to the ...? A) heart and lungs B) brains C) spinal cord D) hips”. Children received one point per correct answer and could thus receive 8 points per posttest on retention knowledge. The transfer questions were open‐ended questions, for example, “What would happen if the bones of a bird were not hollow inside?” The questions were scored with 0, 1, or 2 points by the first author according to a scoring card. Children could thus receive 8 points per posttest on transfer knowledge.

To ensure the reliability of the posttests, a pilot study was performed before conducting the present research. All posttests were administered in approximately 40 children, divided over three schools (Grade 5). After the pilot, the questions were adapted based on their means and corrected item–total correlation. If the mean was not between 0.4 and 0.8 or if the corrected item–total correlation was lower than 0.3, it was adapted. Out of the 48 retention questions, 36 (75%) were improved, and out of the 24 transfer questions, nine (38%) were improved. The alpha of the posttests was .82, indicating good reliability.

Learning time

Learning time was defined as the time (in minutes) children spent studying a multimedia lesson, as was extracted from the log data of the multimedia lessons from the timestamp of the last slide.

Data analyses

To answer the research questions, general linear model repeated‐measures analyses of covariance were conducted. First, the modality effect was examined, with time (short term or long term) and condition (text or audio) as within‐subject‐factors and with group (dyslexia or typically developing) as the between‐subject factor. Verbal and visual working memory were added as covariates. Second, a similar analysis was conducted for the redundancy effect, but with the conditions text, audio, and text and audio. Simple contrasts were performed with the text‐and‐audio condition as a reference category, as we wanted to compare text versus text and audio and audio versus text and audio. Both the modality and redundancy analyses were performed separately for retention and transfer knowledge.

Then, similar analyses were performed to examine the time children spent in the various conditions (learning time). First, the analysis was performed for the modality effect, with learning time (text or audio) as the within‐subject factor and group (dyslexia or typically developing) as the between‐subject factor. Second, the analysis was performed for the redundancy effect, with learning time (text or audio or text and audio) as the within‐subject factor and group (dyslexia or typically developing) as the between‐subject factor. Similar to before, simple contrasts were performed with the text‐and‐audio condition as a reference category. These analyses on learning time also included verbal and visual working memory as covariates.

Due to illness, five children missed one of the six questionnaires (all long term): one child (dyslexia) in the audio condition and four (one child with dyslexia and three typically developing) in the text‐and‐audio condition. According to Elliott and Hawthorne (2005), performing a repeated analysis with a listwise deletion is an inefficient missing‐data method. They argue that the best way to deal with missing data in repeated measures is to substitute a missing value with an average value. The five missing values were thus replaced by the group means (dyslexia or typically developing).

RESULTS

Descriptive statistics

As an extra check on the dyslexia statement, we compared the children with and without dyslexia on general non‐verbal intelligence and reading ability with independent‐samples t tests. In line with their diagnosis, children with dyslexia did not differ on general non‐verbal intelligence compared with the typically developing children in this study, also not after controlling for age differences, but as expected, they did score significantly lower on word reading and pseudo‐word reading (see Table 1).

Table 1

Descriptive statistics for children's general non‐verbal intelligence, word decoding, pseudo‐word decoding, verbal and visual working memory, and learner time per condition per group

	Dyslexia			Typically developing			t	d
	N	M	SD	N	M	SD
General non‐verbal intelligence—raw scores	26	40.23	6.00	38	42.03	6.77	1.09	0.27
General non‐verbal intelligence—percentile score (controlled for age)	26	52.31	26.99	35	58.71	28.03	0.90	0.23
Reading ability
Word decoding	26	45.27	11.91	38	68.84	9.48	8.80***	2.49
Pseudo‐word decoding	26	19.73	5.33	38	36.21	7.67	9.49***	2.15
Working memory
Verbal working memory	26	3.62	1.13	38	5.13	1.30	4.82***	1.24
Visual working memory	26	8.38	2.74	38	10.18	5.01	1.85+	0.45
Time multimedia lessons
Time text condition	17	8.32	4.38	29	4.57	1.34	3.45***	1.16
Time audio condition	17	4.90	0.33	29	4.95	0.82	0.21	0.08
Time text and audio condition	17	5.00	1.15	29	5.03	1.21	0.07	0.03

Note. Although we used standardized scores for the analyses of working memory, we report the sum scores here because the standardized scores by default have M = 0 and SD = 1. Birthdates of three children were unknown, hence the different N in the percentile score of general non‐verbal intelligence. Due to computer malfunction, learning time was only recorded in part of the children, hence the different Ns in the time text, audio, and text‐and‐audio conditions.

p < .10.

p < .05.

p < .01.

p < .001.

In addition, children with dyslexia scored significantly lower on verbal working memory (large effect). With regard to visual working memory, there was no homogeneity of variance, Levene's statistics(1, 62) = 7.70, p = .007, so contrast tests were performed. Children with dyslexia scored marginally significantly lower on visual working memory (small effect).

Also, there was no homogeneity of variance for time spent in the time condition, Levene's statistics(1, 44) = 9.85, p = .003, so contrast tests were performed. Children with dyslexia spent significantly more time in the text condition (large effect). In the other two conditions, children with dyslexia did not differ in the amount of time they spent in the audio or combined text‐and‐audio condition, compared with typically developing children (see Table 1).

The means and standard deviations for the different conditions on the short and long terms for both retention and transfer knowledge, separately for children with dyslexia and typically developing children, are provided in Table 2. Both in all children together and in children with dyslexia and typically developing children separately, no significant correlations could be observed between the time children spent on learning the multimedia lessons and their retention and transfer knowledge of the lessons (ps > .05).

Table 2

Means and standard deviations over time, per condition and group

		Dyslexia				Typically developing
		Short term		Long term		Short term		Long term
	Condition	M	SD	M	SD	M	SD	M	SD
Retention	Text	5.65	1.65	4.96	1.82	5.79	1.42	5.03	1.55
	Audio	5.62	1.70	5.08	1.55	5.53	1.47	4.82	1.61
	Text and audio	5.58	1.58	5.20	1.83	5.89	1.57	5.37	1.36
Transfer	Text	4.27	2.01	3.62	2.12	4.37	1.68	3.74	1.66
	Audio	4.19	1.81	4.00	1.63	4.00	1.77	4.00	1.74
	Text and audio	3.73	2.15	3.60	1.65	4.63	1.84	3.97	1.65

Note. N dyslexia = 26, N typically developing = 38.

Modality effect

Modality effect—retention

Analysis of the retention knowledge with verbal and visual working memory as covariates showed a significant decrease in scores over time, F(1, 60) = 15.79, p < .001, η2 p = .208. Children could recall less information after a week compared with directly after the lessons. No significant main effects were found on condition, F(1, 60) = 0.09, p = .765, η2 p = .002; group, F(1, 60) = 0.001, p = .976, η2 p < .001; verbal working memory, F(1, 60) = 0.07, p = .798, η2 p = .001; or visual working memory F(1, 60) = 0.07, p = .794, η2 p = .001. No two‐ or three‐way interactions were observed (ps > .10).

Modality effect—transfer

Analysis of the transfer knowledge, with verbal and visual working memory as covariates, showed no significant main effects were found over time, F(1, 60) = 2.62, p = .111, η2 p = .042, on condition; F(1, 60) = 0.16, p = .693, η2 p = .003; on group, F(1, 60) = 0.23, p = .634, η2 p = .004; on verbal working memory, F(1, 60) = 0.72, p = .400, η2 p = .012; or on visual working memory F(1, 60) = 0.10, p = .757, η2 p = .002. No two‐ or three‐way interactions were observed (ps > .10).

Modality effect—learning time

Analysis of the amount of time children spent on learning the multimedia lessons, including verbal and visual working memory as covariates, showed a significant main effect of condition, F(1, 42) = 13.15, p = .001, η2 p = .238. Children spent significantly more time on learning in the text condition than in the audio condition. Also, a significant main effect of group was observed, F(1, 42) = 12.84, p = .001, η2 p = .234. Children with dyslexia spent more time learning than typically developing children did. No significant main effects were found on verbal working memory, F(1, 42) = 0.54, p = .466, η2 p = .013, or visual working memory, F(1, 42) = 0.09, p = .77, η2 p = .002.

A significant interaction effect between condition and group was found, F(1, 42) = 13.68, p = .001, η2 p = .246. To interpret the interaction effect further, the analysis was performed separately for children with and without dyslexia. These analyses showed that children with dyslexia spent significantly more time in the text condition than in the audio condition, F(1, 14) = 6.80, p = .021, η2 p = .327, pairwise comparisons p = .006, whereas typically developing children spent an equal amount of time in both conditions, F(1, 26) = 1.65, p = .211, η2 p = .060, pairwise comparisons p = .194. No further two‐way interactions were observed (p > .05).

Redundancy effect

Redundancy effect—retention

Analysis of the retention knowledge, with verbal and visual working memory as covariates, showed a significant decrease in scores over time, F(1, 60) = 24.29, p < .001, η2 p = .288. Children could recall less information after a week compared with directly after the lessons. No significant main effects were found on the redundancy effect: text condition versus combined text‐and‐audio condition, F(1, 60) = 1.13, p = .292, η2 p = .019, or audio condition versus combined text‐and‐audio condition, F(1, 60) = 1.62, p = .208, η2 p = .026. Further, no significant main effects were found on group, F(1, 60) = 0.04, p = .835, η2 p = .001; verbal working memory, F(1, 60) = 0.01, p = .924, η2 p < .001; or visual working memory F(1, 60) = 0.02, p = .889, η2 p < .001. No two‐ or three‐way interactions were observed (ps > .10).

Redundancy effect—transfer

Analysis of the transfer knowledge, with verbal and visual working memory as covariates, showed a significant decrease in scores over time, F(1, 60) = 4.76, p = .033, η2 p = .073. Children could recall less information after a week compared with directly after the lessons. No significant main effects were found on the redundancy effect: text condition versus combined text‐and‐audio condition, F(1, 60) = 0.03, p = .874, η2 p < .001, or audio condition versus combined text‐and‐audio condition, F(1, 60) = 0.35, p = .556, η2 p = .006. Further, no significant main effects were found on group, F(1, 60) = 0.11, p = .737, η2 p = .002; verbal working memory, F(1, 60) = 0.18, p = .671, η2 p = .003; or visual working memory, F(1, 60) = .01, p = .924, η2 p < .001.

With respect to the audio condition versus the combined text‐and‐audio condition, a significant interaction was found between condition and group, F(1, 60) = 7.83, p = .007, η2 p = .115. To interpret this interaction effect further, the analysis was performed separately for children with and without dyslexia. These analyses showed that children with dyslexia performed similar in the audio and the combined text‐and‐audio condition, F(1, 23) = 2.42, p = .133, η2 p = .095,while typically developing children differed in scores on the audio condition compared with the combined text‐and‐audio condition, F(1, 35) = 4.24, p = .047, η2 p = .108. However, pairwise comparisons of the latter group showed no significant difference between the audio condition and the combined text‐and‐audio condition (p = .670). To sum, there seemed to be indications for differences between the audio condition and the combined text‐and‐audio condition in typically developing children, but deeper analysis did not show significant differences. Further, no two‐ or three‐way interactions were observed (ps > .05).

Redundancy effect—learning time

Analysis of the amount of time children spent on learning in the multimedia lessons, including verbal and visual working memory as covariates, showed a significant main effect of condition for the text condition versus the combined text‐and‐audio condition, F(1, 42) = 10.93, p = .002, η2 p = .206, but not for the audio condition versus combined text‐and‐audio condition, F(1, 42) = 0.31, p = .582, η2 p = .007. Also, a significant main effect of group was observed, F(1, 42) = 8.92, p = .005, η2 p = .175. No significant main effects were found on verbal working memory, F(1, 42) = 0.22, p = .642, η2 p = .005, or visual working memory, F(1, 42) = 0.10, p = .755, η2 p = .002.

With respect to the text condition versus the combined text‐and‐audio condition, a significant interaction effect between condition and group was found, F(1, 42) = 14.15, p = .001, η2 p = .252. To interpret the interaction effect between condition and group further, the analysis was performed separately for children with and without dyslexia. These analyses showed that children with dyslexia spent significantly more time in the text condition than in the combined text‐and‐audio condition, F(1, 14) = 7.40, p = .017, η2 p = .346, pairwise comparisons p = .026, whereas typically developing children spent significantly more time in the combined text‐and‐audio condition than in the text condition, F(1, 26) = 3.14, p = .088, η2 p = .108. Pairwise comparisons of the latter group showed no significant difference between the text condition and the combined text‐and‐audio condition (p = .258). To sum, children with dyslexia spend more time in the text condition than in the combined text‐and‐audio condition, whereas in typically developing children, there is no difference between the conditions. Further, no two‐way interactions were observed (ps > .10).

DISCUSSION

In the present research, we aimed to find an optimal multimedia environment for children with dyslexia. Multimedia offers various opportunities to help children with dyslexia; however, it is not clear yet how it can be used in an optimal way to support both study time and knowledge gain, thus leading to efficient learning. Therefore, children with dyslexia and a control group were provided with several multimedia lessons in three conditions: pictorial information presented with (a) written text, (b) audio, or (c) combined text and audio. This way, it was examined to what extent the modality and redundancy effects had an impact on study time and knowledge gain in children with dyslexia and to what extent individual differences in children's working memory capacity was related to these effects.

Children with dyslexia showed weaker working memory capacities compared with typically developing children. With regard to study time, we found modality and reversed redundancy effects on the amount of time children with dyslexia spent in different conditions, whereas in typically developing children, study time was independent of the multimedia environment. Children with dyslexia spent more time in the text condition than in the other two conditions. Concerning knowledge gain, no modality or redundancy effects were found in children with or without dyslexia. In this user‐paced learning environment, children learned as much from pictures with text, audio, or combined text and audio. Working memory did not influence the modality of redundancy effects on study time of knowledge gain.

With respect to study time, partly in line with our first hypothesis, we found modality and reversed redundancy effects on the amount of time children with dyslexia spent in different conditions, but not for typical readers. Children with dyslexia were expected to spend more time in lessons with written text than in lessons with audio, due to slower reading abilities. Our results showed that they spent more time in the written text condition than in the other two conditions: showing a modality effect (text takes longer than audio) and a reversed redundancy effect (text takes longer than text combined with audio). The fact that for children with dyslexia the material in which text was combined with audio did not lead to additional study time (whereas only text did) was not fully in line with our expectation. An obvious explanation could be that children with dyslexia do not really read in a learning environment with both text and audio. An eye‐tracking study would be necessary to confirm this. Considering the fact that study time is relevant, it is important to note that in system‐paced learning environments, study time is a stable factor because it is determined by the system and not the learner and that in research with user‐paced learning environments, study time has often not been taken into account (e.g., Alty et al., 2006; Jamet & Le Bohec, 2007; Scheiter et al., 2014). In the meta‐analyses of Ginns (2005), the only time condition that is indicated is the time spent on the transfer test, not study time itself. In a study by Gerjets et al. (2009), study time was connected to learner control and intuitive knowledge, not to the different multimedia conditions. In a study by Segers et al. (2008), study time was connected to media conditions with no differential effects. However, previous research concerned typically developing children, and not children with dyslexia who read slower. Our results show that study time is an important factor to consider when examining multimedia learning in children with dyslexia. The study time of these children can be reduced, by providing them with audio support.

Concerning knowledge gain, in contrast to the expectations based on the CTML, no differences were found between children with and without dyslexia with respect to modality or redundancy effects. On the basis of CTML, stronger effects would be expected in children with dyslexia, and these differences were expected to be grounded in more difficulties in reading the text and differences in a poorer working memory. Indeed, the children with dyslexia showed weaker working memory capacities, on both the verbal and visual aspects of working memory. However, these differences did not lead to differences in modality or redundancy effects. These results are in line with our alternative hypothesis and research by Mann and colleagues, who did not replicate the modality effect in primary school children. They hypothesized that this was due to the not yet fully developed working memory system in children. Children of 11 years old indeed have a not yet fully developed working memory system, which continues to develop into young adulthood (Gathercole, Pickering, Ambridge, & Wearing, 2004). However, if this argumentation would hold, then we still would have seen differences between children with and without dyslexia due to differences in working memory.

Our results follow the argumentation of Tabbers et al. (2001) that the modality effect does not lie in more efficient use of memory resources. They argued that in system‐paced learning environments, people do not have enough time to relate text and pictorial information, whereas they can listen to a text and look at pictures at the same time. So, in a user‐paced system, where one is in charge of one's own time management, a person can create enough time to switch between text and pictures to optimize his or her learning process. The explanation of the modality and redundancy effects would then be less likely to lie in a working memory overload, but more likely to be a result of a more efficient learning strategy: looking at a picture and listening at the same time to the complementary information (Tabbers et al., 2001). The scattered and scarce research on multimedia learning in children with dyslexia shows no consistent image of this specific group. In adults with dyslexia, Beacham and Alty (2006) showed that optimal conditions for typically developing students are not automatically also optimal conditions for students with dyslexia. However, we did not find differences between the two groups on the modality and redundancy effects on knowledge gain. An explanation could be a variation in learning strategies of children with dyslexia versus controls. Indeed, Kirby, Silvestri, Allingham, Parrila, and La Fave (2008) showed that university students with dyslexia use more time management strategies and reported more often a deep approach to learning than students without dyslexia. These differences can be interpreted as a compensation strategy for the reading difficulties that the students with dyslexia experience, which in turn may drive the lack of differences in the modality and redundancy effects on knowledge gain.

The arguments with regard to working memory also apply to the final hypothesis, as in all children, it was expected that poorer working memory would lead to larger modality and redundancy effects. In contrast to our expectation, we did not find individual differences in working memory that would explain modality or redundancy effects in study time of knowledge gain. It is in line with Witteman and Segers (2010) who also did not find a relation between working memory and the size of the modality effect. Given the theoretical importance of the working memory, this null effect is quite remarkable. Children with poorer working memory were able to use their working memory in a more optimal way in a user‐paced learning environment. Indeed, Ginns (2005) and Tabbers et al. (2004) state that auditory and visual information processing had no impact on learning when children can determine their own pace, which may result in a better integration of audio and visual information. Our study adds to the discussion that individual differences in working memory capacity are not guiding differences in multimedia learning in a user‐paced learning environment. A study on the association between working memory and the size of the modality effect in a system‐paced environment would shed more light on this matter.

On the basis of our results, we can state that study time is an important factor when considering multimedia learning in children with dyslexia. With similar knowledge gain but different study times in the text condition, children with dyslexia need more time to come to an equal amount of knowledge. Study time and knowledge gain were not related: Children with dyslexia simply needed more time to read the whole lesson. Providing them with more study time gave children with dyslexia the opportunity to record all the information. Because children learn as much in both conditions (no redundancy effect on knowledge gain), the combined condition allows children to actively learn in a text condition, without spending too much time purely on reading. Thus, to optimize learning, it is more efficient to provide children with dyslexia with extra audio.

Future research could add to understanding this difference in study time between the text and combined conditions, by using eye tracking to check whether children do read in the combined condition and, if so, whether this read along is different from read only in the text condition. In a similar vein, future research could also examine the role of working memory during learning. Beacham and Alty (2006) argued that the difference between children with and without dyslexia on multimedia learning might lie in the development of compensating strategies in children with dyslexia to compensate for their reading difficulties. Eye tracking would provide the opportunity to examine their possible differences in learning strategies during the lessons. It could also shed light on the cognitive load during multimedia learning, for example, by examining children's cognitive load by pupil dilation. Eye tracking would thus provide information about the learning process. Whereas working memory may explain differences in outcome measures between children in a system‐paced environment, in user‐paced environments, it could be expected that working memory explains differences in process measures, which can be taken into account in future research.

The need for evidence‐based knowledge on multimedia learning in children with dyslexia is urgent. Practitioners need information on how to implement multimedia in an optimal way to support children. We can conclude that in a user‐paced multimedia learning environment, it is more efficient to provide information in an auditory way or with auditory support to children with dyslexia, but not necessary for typical readers.