Predicting Early Literacy: Auditory and Visual Speech Decoding in Deaf and Hard-of-Hearing Children.

The current study investigated the relative contributions of auditory speech decoding (i.e., auditory discrimination) and visual speech decoding (i.e., speechreading) on phonological awareness and letter knowledge in deaf and hard-of-hearing (DHH) kindergartners (Mage = 6;4, n = 27) and hearing kindergartners (Mage = 5;10, n = 42). Hearing children scored higher on auditory discrimination and phonological awareness, with the DHH children scoring at chance level for auditory discrimination, while no differences were found on speechreading and letter knowledge. For DHH children, speechreading correlated with phonological awareness and letter knowledge, for the hearing children, auditory discrimination correlated with phonological awareness. Two regression analyses showed that speechreading predicted phonological awareness and letter knowledge in DHH children only. Speechreading may thus be a compensatory factor in early literacy for DHH children, at least for those who are exposed to spoken language in monolingual or in bilingual or bimodal-bilingual contexts, and could be important to focus on during early literacy instruction.

Auditory speech decoding, or auditory discrimination, is the ability to discriminate units in the continuous flow of speech and is important for spoken language development. Auditory discrimination skills in infancy are related to language skills, such as word production, later on (Kuhl, Conboy, Padden, Nelson, & Pruitt, 2005; Tsao, Liu, & Kuhl, 2004), and are conditional for the development of phonological awareness (Nittrouer, 1996; Studdert-Kennedy, 2002) and letter knowledge (Mann & Foy, 2003). Phonological awareness and letter knowledge, taken as measures for early literacy in the present study, are both strong predictors for later word reading skills in hearing children (Melby-Lervåg, Lyster, & Hulme, 2012). However, for children who are deaf or hard-of-hearing (DHH), access to sound is least limited, and they have been shown to have weaker auditory discrimination and phonological awareness (e.g., Ambrose, Fey, & Eisenberg, 2012; Lederberg, Schick, & Spencer, 2013). This may be one of the reasons why DHH children often experience delays in their word reading, as previous research has shown that phonological awareness skills are also important for word reading in many beginning DHH readers who use spoken language (e.g., Harris, Terlektsi, & Kyle, 2017b; Lederberg et al., 2019). Despite their limited auditory discrimination skills and phonological awareness, letter knowledge in DHH children is often age-appropriate (Ambrose et al., 2012).

Speech may also be represented visually, for example, because some articulatory gestures are visible on ones face. The visual information can be accessed through speechreading. Although visual speech information is accessible, it is not fully specified since not all speech sounds are visually distinguishable (e.g., guttural speech sounds). Speechreading has indeed been found to be associated with word reading abilities in DHH children (Harris, Terlektsi, & Kyle, 2017a; Kyle, Campbell, & MacSweeney, 2016; Kyle & Harris, 2006, 2010, 2011). It is assumed that DHH children use speechreading as a compensatory skill for the development of early literacy, at least to some extent, given the limited possibilities to recognize speech sounds visually. However, the extent to which the ability of speechreading in DHH children is related to early literacy is not yet clear. Moreover, no studies have combined auditory discrimination, speechreading, phonological awareness, and letter knowledge in one design, and their unique influence on early literacy in DHH children thus remains unclear. In the present study, therefore, we will investigate the relative influence of both auditory discrimination and speechreading on phonological awareness and letter knowledge in hearing children and DHH children who are exposed to spoken language, most of them in addition to a sign language.

Auditory Discrimination and Phonological Awareness

Auditory discrimination refers to the discrimination and categorization of speech into units. Speech sounds have to be matched to sound representations in the mental lexicon. For hearing children, one of the first steps in their language development is discriminating between speech sounds. Studies showed that infants already can discriminate between a variety of phonetic features (e.g., Dehaene-Lambertz & Baillet, 1998; Sato, Sogabe, & Mazuka, 2010; Schönhuber, Czeke, Gampe, & Grijzenhout, 2019). Hearing infants thus have the capacity to discriminate speech sounds from early on in their lives.

Auditory discrimination has been shown to be an important predictor for later phonological awareness in hearing children (Janssen, Segers, McQueen, & Verhoeven, 2017; Nittrouer, 1996; Studdert-Kennedy, 2002). Phonological awareness refers to the ability to attend to, reflect upon, and manipulate the different sounds of a word and is a strong predictor of later word reading development in hearing children (Melby-Lervåg et al., 2012). For phonological awareness, one has to be aware of the sound structure of spoken language (e.g., Anthony & Francis, 2005; Goswami, 2001; Webb, Schwanenflugel, & Kim, 2004). Phonological awareness skills develop from the ability to reflect upon or manipulate larger units, such as words, to increasingly smaller units, such as syllables and phonemes (Anthony et al., 2002; Ziegler & Goswami, 2005), and thus builds on auditory discrimination.

For DHH children, auditory discrimination poses an obvious obstacle, as their ability to hear and discriminate speech sounds is at least limited even if digital hearing aids or cochlear implants (CIs) are provided early in life. Several studies showed that deaf infants are not able to discriminate between novel words and familiarized words prior to cochlear implantation but are able to do so within a few months postimplantation (Horn, Houston, & Miyamoto, 2007; Houston, Pisoni, Kirk, Ying, & Miyamoto, 2003). However, deaf infants postcochlear implantation still show less attention to speech stimuli compared to hearing infants. Houston et al. (2003) suggested that this lack of attention to speech by deaf infants results in less well-developed language-specific knowledge, such as language-specific sound contrasts. Fully specified phonological representations of the native language are crucial for developing phonological awareness. Furthermore, Nittrouer and Burton (2001) found that reduced language experience in DHH children leads to lower auditory discrimination skills and phonological awareness abilities compared to hearing children and DHH children with sufficient language experience.

Ambrose et al. (2012) investigated phonological awareness in relation to auditory discrimination in deaf children with CIs and in hearing children, aged 3–5 years. Their phonological awareness measure included phoneme deletion and blending. Auditory discrimination was measured by the children’s ability to discriminate six English speech patterns, for instance, vowel height and consonant voicing. The results showed that the hearing group outperformed the CI group on auditory discrimination and phonological awareness and that phonological awareness correlated with auditory discrimination. In a regression analysis, auditory discrimination did not uniquely contribute to phonological awareness, once language skills and speech production were taken into account. The authors concluded that children with CIs who lag behind hearing children on auditory discrimination (or language skills or speech production) are likely to also fall behind in their phonological awareness development.

Thus, due to their limited and often delayed access to spoken language, and their lower auditory discrimination abilities (Ambrose et al., 2012), DHH children may be at risk for developing underspecified phonological representations, which negatively influences their phonological awareness (Johnson & Goswami, 2010; Lederberg et al., 2013). Even in case of early cochlear implantation, the phonological awareness of a substantial number of deaf children is lower compared to hearing peers (Nittrouer, Sansom, Low, Rice, & Caldwell-Tarr, 2014).

Auditory Discrimination and Letter Knowledge

While there is some discussion on how children learn to read (Bowers, 2020; Bowers & Bowers, 2017; Castles, Rastle, & Nation, 2018), there is strong support that hearing children learn to read by extracting strings of phonemes from speech and match these to their printed versions, graphemes (Ehri, 2005). In alphabetic writing systems, graphemes refer to phonemes in spoken language, and previous research has shown that knowledge of letter sounds and names is a second strong predictor, next to phonological awareness, of later reading development in hearing children (Melby-Lervåg et al., 2012). Research has shown that auditory discrimination correlated significantly with letter name and letter sound knowledge in hearing kindergartners (Mann & Foy, 2003). Another study on hearing children showed that auditory discrimination measured in the first year of kindergarten, predicted letter knowledge in the second year of kindergarten (Van Goch, 2016). Van Goch (2016) concluded that the categorical perception of phonemes is crucial for discriminating letter sounds. In yet another study, speech perception in noise, measured in hearing kindergarteners, predicted letter knowledge in the first grade (Vanvooren, Poelmans, De Vos, Ghesquière, &Wouters, 2017). Auditory discrimination or other speech perception tasks are thus related to the development of letter knowledge.

The DHH children seem to be able to develop age-appropriate letter knowledge (Ambrose et al., 2012; Kyle & Harris, 2011; Werfel, 2017; Werfel, Lund, & Schuele, 2015). Some studies indicated that the development of both letter sound and letter name knowledge in DHH children are not delayed compared to hearing children (Werfel, 2017; Werfel et al., 2015). However, other studies have shown that for DHH children who are learning to read in English, letter sound, and letter name knowledge may develop differently from hearing children, with no delays in letter name knowledge but some delays in letter sound knowledge (Easterbrooks, Lederberg, Miller, Bergeron, & McDonald Connor, 2008; Kyle & Harris, 2011).

Research on the relation between auditory discrimination and letter knowledge in DHH children is scarce. Ambrose et al. (2012) investigated print knowledge in relation to auditory discrimination in deaf children with CIs and hearing children. Print knowledge included letter name and letter sound knowledge. The study found that the groups did not differ on print knowledge and that print knowledge correlated moderately with auditory discrimination in the CI group. However, in a regression analysis, auditory discrimination did not significantly predict print knowledge over and above language skills and speech production. Ambrose et al. (2012) concluded that achieving age-appropriate print knowledge does not require age-appropriate auditory discrimination skills. They speculated that this might be due to early (formal) literacy experiences resulting from instructions in the home literacy environment or in early intervention, as home literacy environments and early literacy skills have been shown to be related (Foy & Mann, 2003).

Nakeva von Mentzer et al. (2013) studied the effects of a letter knowledge intervention (including both letter sounds and letter names) on phonological processing skills, which included an auditory discrimination measure. They showed that DHH children with CIs or hearing aids improved their phonological processing skills as well as their letter knowledge skills after the letter knowledge training. This might indicate that phonological processing skills, including auditory discrimination, and letter knowledge are related in DHH children. However, too strong conclusion cannot be drawn as the study could not identify whether it was the letter knowledge training that improved phonological processing skills, maturation, or test–retest effects.

Still, due to the physical barrier to accessing sound, matching speech sounds to print remains difficult for quite a few DHH children (e.g., Lederberg et al., 2013). That is why research also focuses on studying other ways to promote accurate and fluent early reading, for example, through the systematic use fingerspelling (Lederberg et al., 2019; Miller, Banado-Aviran, & Hetzroni, 2021). Furthermore, it has been suggested that DHH children might use the visual information in print, thus the orthography, to support acquisition of the phonological structure of words (Easterbrooks et al., 2008; Lederberg et al., 2013, 2019).

Speechreading and Phonological Awareness

Speechreading refers to the ability to process speech from visible movements of the face and mouth. From these visual clues, phonological information can be derived, for example, about vowels (e.g., /i/ or /a/) and about certain consonants (e.g., /m/ or /f/). However, several phonemes have overlapping lip patterns (e.g., /m/ and /b/) and several phonemes are not visually distinguishable due to the place of articulation (e.g., /k/ and /g/), entailing that not all phonemes can be discriminated visually. Phonemes with overlapping lip patterns are also referred to as visemes.

Although hearing children do not have to rely on speechreading for access to spoken language as much as DHH children, research has shown that speechreading is also used in communication by hearing individuals (Feld & Sommers, 2009). The influence of visual speech decoding has previously been demonstrated in the McGurk effect in which a mismatch between the visuals and the acoustics is shown, resulting in a different perception of the acoustics, for instance, hearing /ga/ but seeing /ba/ results in hearing /ba/ or /da/ (McGurk & MacDonald, 1976). Even young infants have been reported to integrate auditory speech with its visual part (Burnham & Dodd, 2004; Desjardins & Werker, 2004; Dodd, 1979). Speechreading is thus a natural part of speech processing (Woodhouse, Hickson, & Dodd, 2009). Previous studies by Kyle, Campbell, Mohammed, Coleman, and MacSweeney (2013) and Kyle et al. (2016) have shown no differences on speechreading abilities between DHH and hearing children.

Research has shown that speechreading is related to phonological awareness (Harris et al., 2017a; Kyle & Harris, 2006, 2010). Kyle and Harris (2006) found a concurrent relation between speechreading and phonological awareness, and Kyle and Harris (2010) found that speechreading in DHH 7–8-year-olds predicted phonological awareness 1 year later. Harris et al. (2017a) suggested that speechreading might play an important role in the development of phonological awareness when access to auditory input is reduced, as is the case for DHH children.

The relation between speechreading and phonological awareness has also been investigated in hearing children. Kyle and Harris (2006) found no relation between speech reading and phonological awareness in hearing children. However, in a recent study on speechreading, phonological awareness, and reading in DHH and hearing children, aged 4–8 years, moderate to strong correlations between speechreading and phonological awareness were found for both groups (Buchanan-Worster et al., 2020). Buchanan-Worster et al. (2020) showed that the relation between speechreading and reading was mediated by phonological awareness in both groups, suggesting that speechreading supports the development of phonological awareness. However, it is important to note that a measure of auditory discrimination was not included in that study. It thus remains unclear what the relative contributions of auditory discrimination and speechreading are to phonological awareness.

Speechreading and Letter Knowledge

The relation between speechreading and letter knowledge has not received much attention in the literature, while speechreading has been associated with word reading in DHH children (e.g., Harris et al., 2017a; Kyle et al., 2016; Kyle & Harris, 2006, 2010, 2011). Kyle and Harris (2011) conducted a study on emerging literacy in DHH and hearing children, 5-year-olds. They found that speechreading and letter name and letter sound knowledge did not correlate significantly in both DHH and hearing children. A recent randomized controlled study investigated the effects of a speechreading training on, among other variables, phonological awareness and letter sound knowledge only in DHH children, aged 5–7 years (Pimperton et al., 2019). Children in the experimental group received a training on speechreading, and indeed improved in that ability, but there was no transfer regarding improvement on phonological awareness and letter sound knowledge. Converging evidence thus far indicates no relation between speechreading and letter knowledge in DHH and hearing children, but previous studies did show a relation between speechreading and phonological awareness (Kyle & Harris, 2006, 2010).

The Current Study

The above overview of the literature shows that auditory discrimination and speechreading have each been related to phonological awareness and letter knowledge but have not yet been combined in one study. Furthermore, research in deaf children combined letter name knowledge and letter sound knowledge in one variable. It is thus unclear to what extent auditory discrimination and speechreading are related, and are unique predictors of phonological awareness and letter knowledge in hearing and DHH children, and whether this effect differs between the groups.

The current study, therefore, focused on the relation between auditory discrimination and speechreading in early literacy in DHH, who were exposed to spoken language (and most of them also to sign language), and hearing children. We aimed to disentangle the unique contributions made by auditory discrimination and speechreading on both phonological awareness and letter knowledge, asking the following research questions:

(1) How do DHH and hearing children differ in auditory discrimination and speechreading, phonological awareness, and letter sound knowledge?

(2) To what extent are auditory discrimination and speechreading related and uniquely predict the variance in (2a) phonological awareness and (2b) letter sound knowledge in DHH and hearing children?

Regarding the first research question, we expected DHH children to have weaker auditory discrimination and phonological awareness than hearing children. We did not expect differences between the two groups on speechreading and letter knowledge. As for the second research question, regarding phonological awareness, we expected both auditory discrimination and speechreading to be unique predictors, with speechreading having an additional importance for the DHH group. Regarding letter knowledge, we expected that auditory discrimination predicts letter knowledge in hearing children but not in DHH children. In previous literature, no relation was found between speechreading and letter knowledge. Therefore, we investigated the predictive relation between visual speech decoding and letter knowledge exploratory.

Methods

Participants

In this study, 27 DHH children (17 boys, 10 girls) and 42 hearing children (24 boys, 18 girls) participated. All participating children were in the second year of kindergarten, which is a 2-year program in the Netherlands. Children may enter kindergarten directly after their fourth birthday and formally enroll in the first kindergarten year after the next summer holiday. Children in the second year of kindergarten thus are typically 5–6.5 years old, depending on their date of birth. The mean chronological age of the DHH participants was 6 years and 4 months (SD: 6.38 months) (see Table 1 for an overview of the characteristics of the DHH participants). The DHH children were significantly older than the hearing children, t(67) = −4.44, p < .001, but they had received the same number of years of literacy schooling (which was phonics-based). Three of the DHH children were educated in three different mainstream schools where only spoken Dutch was used. The other 24 DHH children were educated in seven special schools for DHH children in the Netherlands. In all but one special school, bilingual education was practiced, meaning that DHH children are taught in spoken Dutch supported with signs (Sign Support Dutch [SSD]) and Sign Language of the Netherlands (SLN). The other special school taught in spoken Dutch and SSD. In the Netherlands, DHH children go to mainstream education if possible, but when mainstream education cannot provide for the specific requirements of the DHH child, the child attends a special school for deaf education. Approximately, 50% of the DHH children start in special education, and the other 50% starts in mainstream education (Centraal Bureau voor Statistiek, 2016; Van der Ploeg, Wins, &Verkerk, 2020). Our sample of DHH children is thus skewed toward special education.

Table 1

Characteristics of the DHH participants

#ID	Age at testing	Gender	Hearing loss	Hearing devices	Length of use of hearing device	Home language	Level of education reporting parent
1	5;10	Male	Profound	HA + CI	5;9	SSD	College/university
2	5;11	Male	Severe	HA + HA	5;7	SSD	Upper secondary
3	5;7	Female	Severe – profound	HA + HA	5;6	SSD, SLN	Upper secondary
4	5;7	Female	Missing	HA + HA	5;6	SSD, SLN	Upper secondary
5	5;8	Female	Profound	CI + CI	Missing	Dutch	College/University
6	5;9	Male	Profound	CI + CI	4;9	Dutch	College/University
7	5;9	Female	Profound	CI + CI	1;0	SSD	College/University
8	6;1	Male	Mild – profound	HA	1;0	SSD	College/University
9	6;1	Male	Severe – profound	HA + CI	2;9	Arabic	Lower secondary
10	6;1	Male	Profound	CI + CI	4;6	Dutch	Upper secondary
11	6;1	Male	Profound	CI + CI	4;8	Dutch	Lower secondary
12	6;1	Female	Profound	CI + CI	5;3	SSD	College/university
13	6;10	Male	Moderately severe	HA + HA	5;8	Dutch	Lower secondary
14	6;10	Male	Profound	CI + CI	5;8	SSD	Lower secondary
15	6;11	Female	Profound	CI + CI	3;8	Pashto, SSD	Upper secondary
16	6;2	Female	Profound	CI + CI	4;4	SSD	College/university
17	6;3	Male	Profound	CI + CI	3;7	SSD	Lower secondary
18	6;4	Female	Severe	HA + HA	Missing	Dutch	Lower secondary
19	6;4	Female	Profound	CI + CI	Missing	SSD	Upper secondary
20	6;5	Male	Severe – profound	HA + HA	Missing	German, SSD	College/university
21	6;6	Male	Profound	CI + CI	Missing	SSD	Upper secondary
22	6;7	Male	Profound	CI + CI	5;7	Dutch	College/university
23	6;8	Male	Mild – moderately severe	HA + HA	5;9	Dutch, Turkish	Lower secondary
24	6;8	Male	Severe – moderate	BAHA	Missing	Dutch	College/university
25	6;8	Male	Profound	HA + CI	6,1	SSD	College/university
26	6;9	Female	Profound	CI + CI	5;1	English, Telugu	College/university
27	8;1	Male	Profound	CI	0;10	Kurdish, SSD	Unknown

Note. CI = cochlear implant; HA = hearing aid; BAHA = bone-anchored hearing aid; DHH = deaf and hard-of-hearing; SSD = Sign Support Dutch; SLN = Sign Language of the Netherlands.

Concerning the level of education of the reporting parents of the DHH children, 26.9% received lower secondary education, 26.9% received upper secondary education, and 46.2% were educated at college/university level. In all homes, spoken language was used, and in 12 of the homes, spoken language was supported with signs (SSD). In addition to spoken Dutch or SSD, in two homes, SLN was also used. Six of the DHH children were from multilingual spoken language backgrounds. All children used hearing devices: CIs, hearing aids, or bone-anchored hearing aid, with or without contralateral CI or hearing aid.

The mean age of the hearing children was 5 years and 10 months (SD: 3.89 months). The hearing children were recruited from three mainstream schools in the Netherlands. Regarding the level of education of the parents of the hearing children, 7.1% of the reporting parents received lower secondary education, while 21.4% received upper secondary education and 71.1% were educated at college/university. Three hearing children spoke another language at home in addition to Dutch (Arabic, Turkish, or Russian). A Mann–Whitney U test indicated no significant difference between the levels of education of the reporting parent between the DHH group and the hearing group, U = 402, p = .057.

Participants were mainly recruited through special schools. The DHH children who were educated in mainstream schools were recruited through contacting itinerant services, and this proved to be more difficult as schools could not be contacted directly, hence the small amount of mainstream education DHH children in our sample. The hearing children were also recruited through contacting schools. All parents gave active consent. There were no exclusion criteria for participants, as we aimed to recruit an ecologically valid sample. The current study was approved by the Ethics Committee of the Faculty of Social Sciences at the Radboud University (2018–179).

Instruments

Speech decoding

Auditory Discrimination

Auditory speech decoding was assessed via an auditory discrimination test. This task is a subtest of the Testinstrumentarium Taalontwikkelingsstoornissen (Test Instruments Developmental Language Disorders; Verhoeven, Keuning, Horsels, & van Boxtel, 2013). The task comprised 46 minimal pairs of monosyllabic spoken words, which were played on a laptop (see Procedure for more information on the audio setup). The children had to decide whether they heard the same words or two different words (e.g., tak—dak [branch—roof] or man—maan [man—moon])). The reliability of the task was acceptable (Cronbach’s a = .77).

Speechreading

Speechreading was assessed via a task based on the “Words subtest” of the Test of Child Speechreading (Kyle et al., 2013). This speechreading task had a video-to-picture matching design. First, four pictures were shown on a computer screen. The four pictures consisted of the target word and three distractors. Then, a silent video of a female native speaker of Dutch was shown in which the target word was pronounced. After the video was played, the pictures reappeared on the screen. The child then had to point to the matching picture. The task comprised 20 items, the target words represent 25 phonemes, both vowel and consonants, and nine visemes.

The items differed in difficulty; four levels of difficulty were created, each containing five items. The first level of difficulty consisted of the difference between monosyllabic, bisyllabic and trisyllabic word, for example, kam (target [comb]), vrachtwagen (distractor [lorry]), baby (distractor [baby]), and zwembad (distractor [swimming pool]). The second level of difficulty consisted of words with the same number of syllables but without overlapping visemes, for instance, glijbaan (target [slide]), cadeau (distractor [present]), giraf (distractor [giraffe]), and vlieger (distractor [kite]). The third level of difficulty consisted of monosyllabic words in which the target words and distractor words contained one overlapping viseme, for instance, zaag (target [saw]), klok (distractor [clock]), paard (distractor [horse]), and roos (distractor [rose]). The overlapping viseme with the three distractors was either in the onset, the nucleus, or the coda of the words. In the final level of difficulty, the monosyllabic targets and distractors only differed on one viseme, for instance, mond (target [mouth]), maan (distractor [moon]), pop (distractor [doll]), and zon (distractor [sun]). All words were selected from the Streeflijst Woordenschat voor Zesjarigen (Target List Vocabulary for Six-year-olds; Schaerlaekens, Kohnstamm, & Lejaegere, 1999), which is a list with ratings of the percentage of kindergartners expected to know a certain word, as scored by their teachers. All words in the speechreading task were known to at least 76% of the kindergartners based on the Streeflijst Woordenschat voor Zesjarigen (Schaerlaekens et al., 1999). The target pictures and distractor pictures were presented randomized over four locations on the screen. The reliability of the task was acceptable (Cronbach’s a = .79).

Phonological awareness

Phoneme Isolation

Phoneme isolation skill was measured with a task in which the child was asked to sound out the first phoneme of a monosyllabic word (Schaars, Segers, & Verhoeven, 2017), for instance, “With what sound starts ‘soup’?” The correct answer would be /s/. All items were presented orally. All words were CVC-structured. The task consisted of 10 items. The reliability of the task was good (Cronbach’s a = .89).

Phoneme Segmentation

Phoneme segmentation skill was measured with a task in which the child was asked to segment and pronounce the phonemes of a monosyllabic word (Schaars et al., 2017), for example, “Can you break the word ‘nut’ into parts?” The correct answer would be /n-u-t/. The words were presented orally by the researcher. The task consisted of 10 items, increasing in difficulty (from CVC-structured words to CCVC- or CVCC-structured words). The reliability of the task was good (Cronbach’s a = .89).

Rhyme

The rhyme task of the Screeningsinstrument Beginnende Geletterdheid (Diagnostic Instrument for Emerging Literacy; Vloedgraven, Keuning, & Verhoeven, 2009) was administered to the children to assess their passive rhyming skills. The task consisted of two practice items and 15 test items, for instance, “What rhymes with ‘cream’ (/ro:m/in Dutch)? Pictures of soap (/ze:p/in Dutch), tree (/bo:m/in Dutch), and chin (/kIn/in Dutch) were shown. The task was administered through a laptop. The child was presented with three pictures on the screen, which were not named by the researcher, from which they had to choose the picture that rhymed with a word played on the laptop. The reliability of the task was acceptable (Cronbach’s a = .77).

Letter knowledge

To measure letter knowledge, we used a passive phoneme-grapheme mapping task. We tested 22 Dutch graphemes, that is, all graphemes from the alphabet excluding c, q, x, and y, because they are infrequent in the Dutch reading system and are usually not yet introduced in the reading instruction in Grade 1. Children were presented with four graphemes of which they had to choose the grapheme matching the one orally presented by the researcher. The researcher always pronounced the letter sounds. An example of an item is: “Where is the /b/ of ‘boat’?” The children then had to point to the matching grapheme. The three distractors were either phonologically similar, orthographically similar, or were of the same class (vowel or consonant) as the target grapheme. The items were designed in such a way that all graphemes had about the same frequency as distractors.

We chose to test letter knowledge in a passive manner in order to make the task more suitable for children who did not use speech. The graphemes were printed in lower case in four squares on a card. Arial (Monotype, Microsoft) font type of size 180 was used because it is similar to the font used in the reading curricula. The reliability of the task was good (Cronbach’s a = .87).

Procedure

Toward the end of the school year, three individual assessments of about 30 min were conducted. The assessments took place at different days. The assessments are part of a larger longitudinal study on reading development; tasks described in this paper are part of that larger study, but more tasks, not described in this paper, were administered. Tasks were administered by the first author and four research assistants. All research assistants received a training on how to administer the tasks. The tasks were conducted in the same order for all participants (letter knowledge, phoneme isolation, phoneme segmentation, rhyme, speechreading, and auditory discrimination). The researchers provided spoken language instructions for hearing children and DHH children in mainstream settings since the latter ones did not use SSD or sign language. For the DHH children in schools for the deaf, instructions were given in Dutch, supported with signs (SSD). While the instructions for the DHH children were given in SSD, only spoken Dutch was used while administering the task items so as to not elicit responses in sign language but when a response in spoken language was required. For all children, the instructions were repeated if necessary and some help was allowed during the practice items in order to make sure the children understood the tasks. All tasks were administered at the school during school hours and in a quiet room. For 11 DHH children, frequency modulation (FM) systems were used during the individual assessments. The remaining children did not use FM systems during class (n = 11) or equipment was unavailable (n = 5). The face of the researcher was always visible during the orally presented tasks. For the tasks that involved a laptop, the volume of the laptop was set at 70 dB for the hearing children prior to testing. For the DHH children, prior to testing the volume of the laptop was adjusted to a level that was comfortable for them to hear. The maximum of the volume was 75 dB.

Analyses

For the analyses, we used IBM SPSS (version 25). First, we conducted a principal component analysis to create the variable phonological awareness, which included the tasks phoneme isolation, phoneme segmentation, and rhyme. All phonological awareness tasks had high factor loadings; phoneme isolation (.832), phoneme segmentation (.866), and rhyme (.832). To calculate the phonological awareness scores, we have scaled the rhyme task to 10 and summed it with the scores on phoneme isolation and phoneme segmentation.

Second, we inspected the data for normality by conducting one-sample Kolmogorov-Smirnov (K-S) tests. For auditory discrimination, the K-S test indicated a normal distribution D (69) = .10, p = .07, and group differences were tested with a Welch’s t-test. The other tasks did not follow a normal distribution: speechreading D (69) = .11, p = .04, phonological awareness D (69) = .11, p = .03, and letter knowledge D (69) = .15, p < .001. Because of the violation of normality in our sample, we conducted Mann–Whitney U tests to test the group differences for speechreading, phonological awareness, and letter knowledge.

Third, to test the predictive values of auditory discrimination and speechreading on phonological awareness in each group, we conducted separate multiple hierarchical regression analyses per group. We conducted regression analyses as these analyses are robust to violations of normality. For the analysis of letter knowledge, we conducted a multiple regression analysis with 3,000 bootstrap samples, and the bootstrapped 95% confidence intervals and p-values are reported.

Results

Table 2 shows the descriptive statistics for both the DHH children and hearing children. To answer the first research question, concerning the differences between the groups, an independent samples t-test indicated that the hearing children scored higher on the auditory discrimination task than the DHH children, t (66.71) = 7.42, p = < .001, d = 1.76. The DHH children scored below chance on the auditory discrimination task. The hearing children also scored higher on phonological awareness than the DHH children, U = 187.50, p = < .001, d = 1.36 (also on all individual phonological awareness tasks, see Table 2). No differences were found on letter knowledge, U = 438.50, p = .11, d = 0.39, and speechreading, U = 445, p = .09, d = 0.37 (mean rank of DHH children = 40.07; mean rank of hearing children = 31.74).

Table 2

Descriptive statistics for DHH children (n = 27) and hearing children (n = 42)

Task	DHH children				Hearing children
	m	SD	Mdn	Range	m	SD	Mdn	Range	Min–max	p
Auditory discrimination	22.22	3.76	22	15–29	30.48	5.48	31.50	19–42	0–46	<.001
Speechreading	11.85	5.17	13	1–20	10.07	3.32	10	4–18	0–20	.09
Phonological awareness	11.57	5.83	12	2.67–24	19.55	5.73	21	3.33–29.33	0–30	<.01
Phoneme isolation	5.18	3.52	6	0–10	7.59	2.52	8.50	1–10	0–10	<.001
Phoneme segmentation	.85	1.88	0	0–7	3.86	2.90	4	0–10	0–10	<.001
Rhyme	8.19	2.59	8	4–14	12.14	2.49	13	2–15	0–15	<.001
Letter knowledge	17.85	4.63	20	6–22	16.74	4.14	18	5–22	0–22	.11

DHH = deaf and hard-of-hearing.

Table 3 shows the correlations for the DHH children and hearing children. In the DHH group, speechreading correlated strongly to phonological awareness and letter knowledge, and phonological awareness correlated strongly to letter knowledge. For the hearing children, auditory discrimination had a weak but significant correlation to phonological awareness and phonological awareness correlated moderately to letter knowledge.

Table 3

Correlations for DHH and hearing children

	1	2	3	4
1. Auditory discrimination	—	.28	.31*	.11
2. Speechreading	−.02	—	.17	.22
3. PA	−.09	.72***	—	.41**
4. LK	.08	.61***	.65***	—

Note. Correlations for DHH children below the diagonal, correlations for hearing children are above the diagonal; PA = phonological awareness, LK = letter knowledge; DHH = deaf and hard-of-hearing.

 < .05; **p < .01; ***p < .001.

Regarding the impact of auditory and visual speech decoding on phonological awareness (research question 2a), a hierarchical regression analysis was conducted separate for each group (see Table 4). The analyses showed that for DHH children, only speechreading predicted phonological awareness, explaining 52% of the variance. For hearing children, neither auditory discrimination nor speechreading explained a significant amount of variance in phonological awareness.

Table 4

Predictors of phonological awareness per group

Independent variable	DHH children						Hearing children
	B	SE B	β	95% CI	p	R2	B	SE B	β	95% CI	p	R2
Auditory discrimination	−.12	.22	−.08	[−.58, .33]	.580	.52*	.30	.17	.28	[−.04, .63]	.081	.10
Speechreading	.81	.16	.72	[.48, 1.14]	<.001		.16	.27	.09	[−.40, .71]	.572

Note. DHH children Adj. R final model = .48; Hearing children Adj. R final model = .06. DHH = deaf and hard-of-hearing.

* p < .001.

To test the impact of auditory discrimination and speechreading on letter knowledge (research question 2b), per group, another hierarchical multiple regression analysis was conducted (see Table 5). The analyses showed that for DHH children speechreading predicted letter knowledge, explaining 38% of the variance in speechreading. Auditory discrimination did not have a significant effect on letter knowledge. For the hearing children, the model did not explain a significant amount of variance.

Table 5

Predictors of letter knowledge per group

Independent variable	DHH children						Hearing children
	B	SE B	β	95% CI	p	R2	B	SE B	β	95% CI	p	R2
Auditory discrimination	.11	.20	.09	[−.30, .46]	.584	.38*	.04	.12	.06	[−.25, .35]	.785	.05
Speechreading	.55	.14	.61	[.15, .82]	.007		.26	.20	.21	[−.15, .67]	.099

Note. DHH children Adj. R final model = .33; Hearing children Adj. R final model = .004. DHH = deaf and hard-of-hearing.

* p = .003.

Discussion

The goal of the present study was to investigate the relative contributions of auditory speech decoding (i.e., auditory discrimination) and visual speech decoding (i.e., speechreading) on phonological awareness and letter knowledge in DHH and hearing children. The group of DHH children were mainly from special education, all used hearing amplification and all children were exposed to spoken language, some supported with signs or additionally SLN. The results show that DHH children scored lower on auditory discrimination and phonological awareness than hearing children, but the groups did not differ on speechreading and letter knowledge. Furthermore, phonological awareness and letter knowledge were uniquely predicted by speechreading and not by auditory discrimination in DHH children. Phonological awareness and letter knowledge were not predicted by auditory discrimination or speechreading in hearing children.

Regarding the first research question, the results are in line with our hypothesis: DHH children had weaker scores on auditory discrimination and phonological awareness than hearing children but did not differ on speechreading and letter knowledge. This is in line with previous research (e.g., Ambrose et al., 2012; Kyle & Harris, 2011).

Concerning the second research question, regarding phonological awareness, auditory discrimination did not significantly predict phonological awareness in the DHH group or in the hearing group, which is contrary to our hypothesis. In the study by Ambrose et al. (2012), auditory discrimination and phonological awareness were related in deaf children with CIs: Although oral language abilities, which consisted of speech perception, speech production, language expression, language comprehension, and vocabulary, together predicted phonological awareness, not one of the predictors explained a unique amount of variance in phonological awareness. An explanation for their different results could be task related; in the auditory discrimination task used in Ambrose et al. (2012), children had to respond to differences in a string of repeated vowel-consonant-vowel utterances, for instance, /u:du:/ versus /a:da:/. Six different English speech contrasts (vowel height and place, consonant voicing, manner, and place) were tested. Our task involved more than six Dutch speech contrasts and furthermore consisted of minimal pairs, making the presented differences smaller than in the task used by Ambrose et al. (2012). The task used in the present study was thus more fine-grained and therefore more difficult for DHH children. The level of task difficulty could explain the difference in results between the current study and the study by Ambrose et al. (2012). Maybe due to the difficulty of our auditory discrimination measure, auditory discrimination did not predict phonological awareness in the DHH group, as previous studies have shown that DHH children with better auditory access, who also use spoken language, seem to have better phonological awareness (Antia et al., 2020; Lederberg et al., 2019). Regarding the hearing group, although auditory discrimination was not predictive of the phonological awareness in the regression analysis that also included speechreading, the two skills did correlate significantly. This is in line with previous research (e.g., Janssen et al., 2017).

As for speechreading, we hypothesized that it would be of great importance to phonological awareness in the DHH group. Indeed, our results show that speechreading was a strong predictor for phonological awareness in the DHH group, explaining 52% of the variance. Previous research has suggested this link between speechreading and phonological awareness (e.g., Harris et al., 2017a), but the present study is one of the first to show the concurrent relation between speechreading and phonological awareness in DHH children. We did not find a predictive relation between speechreading and phonological awareness in the hearing children, which is in line with the results of Kyle and Harris (2006). We assume that, although DHH and hearing children did not differ on speechreading, hearing children do not have to rely as much on speechreading as DHH children, as they have full access to auditory input, and they do not need the visual input to develop phonological awareness.

As for the second research question, regarding letter knowledge, auditory discrimination did not correlate with letter knowledge in the DHH group, which is fully in line with our hypothesis and similar to the findings by Ambrose et al. (2012). Ambrose et al. (2012) concluded that, for DHH children, having well-developed letter knowledge does not require well-developed auditory discrimination skills and that age-appropriate performance on at least some early literacy tasks is positive for later reading development. However, in the study by Nakeva von Mentzer et al. (2013), DHH children did improve on an auditory discrimination measure after phoneme-grapheme correspondence training, but the study could not conclude whether the training caused the improvement. Furthermore, the study did not analyze whether initial auditory discrimination could predict letter knowledge. Therefore, the relation between auditory discrimination and letter knowledge in DHH children needs to be investigated further. For hearing children, we expected that auditory discrimination would be associated with letter knowledge, but our results showed no significant correlation. This is not in line with previous findings by Mann and Foy (2003) and Van Goch (2016). An explanation could be that, in the current study, the relation between auditory discrimination and letter knowledge might be indirect via phonological awareness. The relation between auditory discrimination and letter knowledge would then be mediated by phonological awareness. The correlation table provides evidence for this explanation, as phonological awareness correlates with both auditory discrimination and letter knowledge. However, further research is needed.

The relation between speechreading and letter knowledge was investigated exploratory. Speechreading only correlated with letter knowledge in the DHH group. The regression analysis also showed that speechreading only predicted letter knowledge in the DHH group, explaining 38% of the variance. Speechreading did not have a significant effect on letter knowledge in the hearing group. These results for the DHH children are not in line with previous findings by Kyle and Harris (2011), although the results for the hearing group are consistent with their findings. In Kyle and Harris (2011), speechreading did not correlate to the letter sound and letter name knowledge in either DHH or hearing children. This difference could be explained by the variation in the number of visemes in each language (Van Son, Huiskamp, Bosman, & Smoorenburg, 1994) and the transparency of the orthography; Dutch has transparent phoneme to grapheme mappings (Verhoeven, 2017), whereas English, as was studied by Kyle and Harris (2011), has more opaque mappings (Perfetti & Harris, 2017). Both speechreading and letter knowledge require visual skills, and both link a visual referent to a phoneme. In an orthography with consistent mappings between phonemes and graphemes, the graphemes have tighter connections to mouth shapes of the phonemes they represent, which could entail that it is thus easier to speechread phonemes and link them to their orthographic counterparts. Furthermore, about half of the children in the study by Kyle and Harris (2011) preferred to communicate through sign language, while all children in our sample communicated at least partly in spoken language and thus possibly relied on speechreading to a greater extent than the children in Kyle and Harris (2011).

It is important to note that speechreading and phonological awareness could be strongly related because in two out of three phonological awareness tasks (phoneme isolation and phoneme segmentation) as well as the letter knowledge task, were administered face to face. This makes it more difficult to disentangle the effects of speechreading on phonological awareness and letter knowledge. A previous study has shown that deaf children with CIs perform better on phonological awareness tasks when these tasks are administered in live, visual speech, indicating that speechreading could aid in completing phonological awareness tasks (Spencer & Tomblin, 2008). However, our results on the relation between speechreading and phonological awareness are similar to the results found in Harris et al. (2017a). In their study, a phonological awareness task that consisted of picture matching was used without using speech. Furthermore, teachers also use speech when giving early literacy instruction, therefore, DHH children can also use speechreading in the classroom.

Our speechreading task was based on the task developed by Kyle et al. (2013). In both our study and the study by Kyle et al. (2013), DHH children and hearing children did not differ on their speechreading abilities. The task by Kyle et al. (2013) also included words, sentences, and short stories, whereas our task only included words. Different levels of speechreading could be differently related to early literacy, word reading, and reading comprehension and could differ between DHH and hearing children, as was found in Kyle et al. (2016). It would be interesting for future studies to investigate this further and also to include sentences and short stories in a Dutch speechreading study, because in everyday conversations, words rarely occur in isolation. To include sentences and short stories, as in the task developed in Kyle et al. (2013), would make the Dutch task more ecologically valid.

Our study is one of the first studies to show that for DHH children who use (some) spoken language, speechreading is also important to the stage prior to reading, namely early literacy (phonological awareness and letter knowledge). Speechreading can thus be a compensatory factor in early literacy, as it predicts both phonological awareness and letter knowledge. Future studies should explore the effect of speechreading on other early literacy measures such as rapid naming and verbal short-term memory. Our study also shows that speechreading could be of great importance for DHH children in gaining access to spoken language phonology. Previous studies have focused on the relation between speechreading and reading (Kyle & Harris, 2006, 2010, 2011; 2016) and on the mediating role of phonological awareness between speechreading and reading (Buchanan-Worster et al., 2020) but did not investigate differences in the relative contribution of auditory and visual speech decoding, or auditory discrimination and speechreading, to phonological awareness and letter knowledge. Our results thus extend previous findings, combining the influence of auditory and visual speech decoding on early literacy and showing the importance of speechreading in phonological awareness and letter knowledge in DHH children.

Limitations and Future Directions

One limitation of the current study is the fact that the sample included more DHH children from special education compared to mainstream education. This limits the generalizability of our findings. However, in our sample, all DHH children were exposed to spoken language at home, and we expect that DHH children in mainstream education will also use mostly spoken language at home and at school. Future studies investigating the role of speechreading in early literacy in DHH and hearing children should include a more balanced sample between special education and mainstream education when possible. Furthermore, it would also be informative to compare groups of DHH children differing on their preferred language modality (spoken language only, bimodal or sign language only), as subgroups could develop (early) literacy differently (Antia et al., 2020; Lederberg et al., 2019). Most DHH children in our sample were exposed to SSD and SLN at school. However, it is unclear how SSD or SLN were used during early literacy instruction because these data were not collected. We therefore cannot determine the influence of a possible additional (visual) language on early literacy development in DHH children. Future studies could investigate whether an additional visual language is beneficial for early literacy instruction for DHH children. Furthermore, it would also be insightful to compare groups of DHH children differing in exposure to sign language and the effects on early literacy, as using sign language during early literacy instruction might increase the comprehension of phonological awareness and letter knowledge and therefore promote early literacy instruction.

A second limitation is the cross-sectional design of our study. Future studies should investigate the role of auditory and visual speech decoding on early literacy longitudinally, because with a longitudinal design, causal effects can be determined. Furthermore, a longitudinal study should also investigate the relation between phonological awareness and letter knowledge. There is general consensus that these skills are reciprocally related (e.g., Burgess & Lonigan, 1998), and our results also show that phonological awareness and letter knowledge were related in both groups. Furthermore, the reciprocal effect might be different for DHH children and hearing children, as previous studies suggest that DHH children develop their phonological awareness through letter knowledge or reading instruction (Musselman, 2000), as graphemes provide a visual mnemonic for an auditory phoneme (Beal-Alvarez, Lederberg, & Easterbrooks, 2012).

Clinical Implications

Notwithstanding the limitations, our study has important implications for practice. First, our study has shown that speechreading is an important skill for DHH children who are also exposed to spoken language and who are educated in special education to gain access to early literacy, both phonological awareness and letter knowledge. Therefore, it seems important to focus attention on speechreading during early literacy instruction. Instructors should be aware that their face is clearly visible for the DHH children while giving literacy instruction, so children can match phonemes and graphemes to their corresponding lip pattern. Training of speechreading might seem indicated even if a recent study on the effects of speechreading training in DHH children failed to show significant or lasting effects on phonological awareness, letter knowledge, or reading (Pimperton et al., 2019). However, in this study by Pimperton et al. (2019), participants only received 8 hr of speechreading training during 12 weeks, which may not have been enough to foster effects on literacy. Also, this speechreading training program was not embedded in a broader literacy program. And finally, since this study was conducted with respect to literacy in English, it is quite possible that speechreading training could yield better results in languages with more transparent orthographies, such as Dutch. More research in this domain is needed to investigate how speechreading can impact early literacy development in DHH children.

Second, despite having lower auditory discrimination and phonological awareness skills, DHH children seem to attain age-appropriate letter knowledge. Even DHH children who experience difficulties in auditory discrimination are able to develop letter knowledge that is on par with hearing children. As letter knowledge is an important predictor for later reading, this could be of importance to compensate for their weak phonological awareness. Not only formal instruction in school could aid in developing letter knowledge but also the home literacy environment could be of great support (Burgess, Hecht, & Lonigan, 2002).

Finally, our study points out that not all DHH children who use hearing aids or CIs from an early age on have sufficient access to auditory speech to enable accurate auditory discrimination. This implies that even with modern hearing devices, DHH children miss a considerable amount of the auditory input. It emphasizes the importance of the visual route to language, that is, through sign language, fingerspelling, or speechreading. Parents, teachers, and other professionals should be aware that, although DHH children may have some functional hearing and are exposed to spoken language, this might not be enough to develop auditory discrimination or phonological awareness skills.

Conclusions

It can be concluded that, for DHH children enrolled in schools for deaf children, speechreading can be helpful to their development of early literacy. Those children who had better speechreading skills also had better phonological awareness and letter knowledge. For these DHH children, speechreading thus functions as a compensatory skill for developing early literacy. Furthermore, auditory discrimination is still a very difficult task despite early provision of hearing aids or CIs.