Orthographic influences when processing spoken pseudowords: theoretical implications.

When we hear an utterance, is the orthographic representation of that utterance activated when it is being processed? Orthographic influences have been previously examined in relation to spoken pseudoword processing in three different paradigms. Unlike real word processing, no orthographic effects with pseudowords have been observed in a phoneme goodness ratings task, and there is a mixed outcome in studies looking for spelling-sound consistency effects. In contrast, the orthography of spoken pseudohomographs has been shown to be activated, given that they prime their homographic base word. Explanations are sought for the findings in these three paradigms, leading to an exploration of theoretical models of spoken word recognition.

There is now considerable evidence that orthographic information is activated during the processing of verbal utterances, whether this automatically occurs during speech recognition (e.g., Slowiaczek et al., 2003; Chéreau et al., 2007; Taft et al., 2008; Peereman et al., 2009) or is controlled strategically (e.g., Cutler et al., 2009). Inconsistencies arise in the literature, however, in relation to the influence that orthography has on the processing of non-existing words (i.e., “pseudowords”). If orthography were automatically activated, one might expect its influence to be observed whatever the type of utterance being processed, whether it be a real word or a pseudoword. Even if orthography were not automatically activated, it would be expected that failure to access a lexical representation on the basis of purely phonological information would provide the likely circumstances under which orthography might come into play, and this means that any orthographic effect would be most likely observed when the utterance is a non-existing word. However, despite finding orthographic effects in the processing of spoken words, Ziegler and Ferrand (1998), Ventura et al. (2004), and Cutler et al. (2009) found no equivalent effects for spoken pseudowords. In contrast, Pattamadilok et al. (2007) and Pattamadilok et al. (2009) found effects for both real words and pseudowords, while the orthographic effects observed by Taft et al. (2008) arose from pseudoword utterances. The purpose of the present paper, therefore, is to explore the circumstances under which effects are or are not observed with pseudowords and examine how this might contribute to the theoretical account of orthographic involvement in spoken word recognition.

Experimental Findings

Orthographic consistency effects in auditory lexical decision

The most commonly adopted technique for examining orthographic effects in auditory lexical processing has been the comparison of lexical decision responses to monosyllabic utterances whose rime is either consistently spelled or not (e.g., Ziegler and Ferrand, 1998; Ventura et al., 2004; Ziegler et al., 2004, 2008; Pattamadilok et al., 2007, 2009; Perre et al., 2009). For example, the rime of the English word /stIŋ/ (i.e., /Iŋ/) can only ever be spelled ing and the word sling is therefore “consistent.” An example of an “inconsistent” word is /bri:f/ (i.e., brief) because its rime has other spellings, namely, eef and eaf. All of the above studies found longer lexical decision responses to inconsistent words than consistent words using French materials (apart from Ventura et al., 2004, who used Portuguese materials). Of most relevance here, however, is that such a consistency effect was not always found for pseudowords (an English example being /kri:f/ versus /spIŋ/). Pattamadilok et al. (2007, 2009) showed clear consistency effects for pseudowords, but Ziegler and Ferrand (1998) and Ventura et al. (2004) did not. The other studies did not report any pseudoword data.

Phoneme goodness ratings

Cutler et al. (2009) report a completely different experimental paradigm that revealed an orthographic effect for real words, but not for pseudowords. Many tokens of a word like /blεs/ (i.e., bless) or /voIs/ (i.e., voice) were aurally presented, each with their final phoneme varying acoustically from a clear /s/ to a clear /z/. When asked to rate how good the /s/ was for a particular token, participants gave higher ratings for the best tokens of /s/ when it was actually spelled with an s (e.g., bless) than when it was not (e.g., voice). No such orthographic bias was observed for equivalent pseudowords, such as /plεs/ (typically spelled pless) versus /floIs/ (typically spelled floice), where the orthographic form must be sublexically generated. Cutler et al. (2009) therefore conclude that the orthographic influence on phoneme-based judgments emanates from lexical information that does not automatically flow in a top-down manner to sublexical levels. In other words, activation of the orthographic form of a pseudoword is not something that happens automatically during speech processing.

Pseudohomograph priming

The notion that orthography is not automatically generated sublexically during speech processing appears incompatible with the findings of Taft et al. (2008). Auditory lexical decision responses to word targets were shown to be facilitated by the prior utterance of a pseudoword that could be spelled in the same way as the target. For example, presentation of the pseudoword prime /swæp/ facilitated recognition of the target word /swɔp/ (relative to an unrelated pseudoword prime) because the likely spelling of the prime matched the spelling of the target (i.e., swap). This is called “pseudohomograph” priming because /swæp/ is a pseudoword that has the same spelling as a real word, namely, /swɔp/. Controlling for the possibility that such priming arose purely from the phonological similarity of the prime and target, no facilitation was observed when the orthography of the phonologically similar prime and target did not match (e.g., /stæp/-/stɔp/, where stap has a different spelling to stop). The priming conditions of the experiment were set up in such a way that participants were unaware of the orthographic relationship between the prime and target and, for this reason, it was concluded that the pseudohomograph effect arose automatically. Moreover, the automatically activated orthographic information must have been generated sublexically because it was the spelling of a pseudoword that produced the priming effect.

How might the results obtained with pseudowords in the different experimental paradigms be explained?

A Timing-Out Mechanism?

Ziegler and Ferrand (1998) explain their failure to find consistency effects with pseudowords in terms of the idea that the “no” responses in a lexical decision task are made once a threshold for lexical activation is timed out. They argue that such a timing-out mechanism has the potential to obscure any consistency effect for pseudowords. The question then arises as to why Pattamadilok et al. (2007, 2009) were able to find effects for “no” responses in lexical decision, and the answer might lie in the fact that the difference they observed between word and non-word responses (i.e., the lexicality effect) was greater (204 and 136 ms for the two studies respectively) than the 75-ms effect observed by Ziegler and Ferrand (1998). In other words, for whatever reason, the participants tested by Ziegler and Ferrand might have timed out their responses before any impact of consistency emerged, unlike the participants of Pattamadilok et al. However, if the consistency effect arises at the sublexical level, as Ziegler and Ferrand suppose, it needs to be explained why an increase in the time spent waiting for a lexical representation to reach threshold allows such a sublexical effect to emerge. It could even be argued that the longer it takes to respond to a non-word, the lower the accuracy in timing out the response and, hence, the greater the opportunity to wash out any consistency effect. In addition, the task used by Cutler et al. (2009) was not lexical decision and did not involve a timing-out mechanism, yet it showed no orthographic effect for pseudowords.

A Lexical Locus for the Effect?

Another reason that can be given for the lack of an orthographic influence in spoken pseudoword processing is that such an influence arises from the lexical level of representation (Ventura et al., 2004; Cutler et al., 2009). Pseudowords, by definition, have no whole-word representation in lexical memory and, therefore, if orthography is associated with phonology solely at the lexical level, its only possible impact on a pseudoword will be when that pseudoword partially activates an existing lexical representation. Such partial activation will occur when the pseudoword is phonologically similar to a real word, for example, when they share a rime. So, the inconsistent English pseudoword /kri:f/ would partially activate real words that share the rime /i:f/, in turn activating a range of different orthographic forms (e.g., brief, reef, leaf) that can compete with each other to generate the consistency effect. Pattamadilok et al. (2007, 2009) argue that this is the source of the orthographic consistency effect they observe for pseudowords, and that the fewer lexical neighbors in Portuguese than French means that there is not enough partial lexical activation in the former language to show an effect with pseudowords (Ventura et al., 2004). However, such a cross-language account cannot explain why Ziegler and Ferrand (1998) also failed to find an effect with French materials.

In order to explain the results of Taft et al. (2008), it can presumably be argued that a sublexical route from phonology-to-orthography does eventually come into play, in the same way that the sublexical route from orthography-to-phonology is claimed to lag behind the lexical route from orthography-to-phonology in reading aloud (e.g., Coltheart et al., 2001). Given that there was a considerable delay between the presentation of the prime and target in the Taft et al. (2008) study, there would be plenty of time for the orthography of the pseudoword prime to be generated sublexically and hence activate the relevant lexical representation via this orthographic form.

There is a problem, however, with the distinction drawn between lexical and sublexical pathways, and that is that the consistency effect actually centers on a sublexical unit, namely, the rime of the word. The lexical representation for the inconsistent word /bri:f/ competes with that of /ri:f/ and /li:f/ because of the differing spelling of their rime (i.e., ief, eef, and eaf respectively). If the rime were not important as a unit of processing, brief would be no more in conflict with reef and leaf than it would be with brick, self, or any other word overlapping in structure. What is relevant is that the rime /i:f/ maps onto multiple orthographic forms, in contrast to a consistent rime, like /Iŋ/. So, the fact that lexical processing actually involves sublexical structures means that the distinction between lexical and sublexical pathways is blurred. Indeed, in dual-route models of visual word recognition, spelling-to-sound consistency effects are either unaccounted for (e.g., Coltheart et al., 2001) or arise from the sublexical pathway whereby they are observed for both words and pseudowords (e.g., Perry et al., 2007). Therefore, a dual pathways account of spoken word recognition cannot ascribe sound-to-spelling consistency effects to the lexical pathway as proposed by Pattamadilok et al. (2007, 2009), and such effects must arise instead from a sublexical pathway. This then means that orthographic effects should be observed with pseudowords as well as words, which is what Pattamadilok et al. (2007, 2009) observed. However, such a conclusion leaves the findings of both Ventura et al. (2004) and Ziegler and Ferrand (1998) unexplained.

Matching of Materials

The two studies that failed to observe a consistency effect for pseudowords (i.e., Ziegler and Ferrand, 1998; Ventura et al., 2004) actually provide very few details about their pseudoword items and, therefore, it is possible that their materials were not as well controlled as in the studies by Pattamadilok et al. (2007, 2009) where the consistent and inconsistent pseudowords were more carefully matched. Indeed, Ziegler and Ferrand (1998) report that their consistent pseudowords were over 80 ms longer in duration than their inconsistent pseudowords, and that stimulus duration had to therefore be treated as a covariate in the analysis. Ventura et al. (2004) used a similar statistical approach to handle mismatched stimulus duration in their first experiment, a less-than-ideal way of controlling for that factor, but they did match the duration of the consistent and inconsistent items in their second experiment.

However, in neither study is it said whether other important factors, such as similarity of the pseudoword to a real word, were matched between the two conditions. The phonological neighborhood of the pseudoword items are not reported (a measure of similarity to real words, cf. Luce and Pisoni, 1998). If the consistent pseudowords happened to be phonemically similar to more real words than were the inconsistent pseudowords, then the relative difficulty in distinguishing the former from the real words in the lexical decision task could have counterbalanced any beneficial effect of orthographic consistency.

Another potentially important factor is the point at which the pseudoword deviates from being a real word. It can be shown, not surprisingly, that the earlier the deviation from a real word, the faster a pseudoword can be identified as not being an existing word (see e.g., Marslen-Wilson, 1987). For example, /spIŋ/ and /spæŋ/ are both consistent pseudowords of similar duration, but the latter might take longer to classify as a pseudoword than the former because, after the /ŋ/ is processed, /spæŋ/ can still go on to become a real word (i.e., spank or spangle), while /spIŋ/ cannot. None of the lexical decision studies, including Pattamadilok et al. (2007, 2009), report whether the consistent and inconsistent pseudowords were matched on their deviation point, ideally by having a group of participants identify the stimuli from increasingly larger fragments (i.e., using a “gating” procedure, cf. Grosjean, 1980; Tyler, 1984) to determine how large the fragment must be before the deviation from a real word is identified. If the two conditions were not well-matched on this factor, it provides a further explanation for the lack of a consistency effect for pseudowords. Indeed, it also needs to be established whether or not the effect observed by Pattamadilok et al. (2007, 2009) can be explained in terms of the consistent pseudowords having an earlier deviation point than the inconsistent words. Therefore, it is unclear at this stage if there really is a pseudoword consistency effect because a comparison of response times to different items across conditions requires that the two conditions be effectively matched.

Matching between conditions, however, is not an issue in the phoneme goodness rating task (Cutler et al., 2009). Therefore, the lack of an orthographic effect for pseudowords in that task cannot be explained in terms of poorly controlled materials.

An Interactive-Activation Explanation

To see how orthographic effects in spoken word recognition might be explained, it is useful to adopt a concrete framework for conceptualizing how information might be represented and processed. The TRACE model of McClelland and Elman (1986) is probably the most influential model of spoken word processing, adopting an interactive-activation architecture with units of phonemic representation being hierarchically organized in terms of grain-size. Such a model, however, has no orthographic component. Instead, then, an interactive-activation framework that includes both orthography and phonology will be used as a way of illustrating how orthographic effects may or may not emerge for spoken pseudowords. The framework is a variant of the model of lexical processing put forward by Taft (1991, 2006a) in relation to reading, adapted to spoken word recognition. The model is depicted in Figure 1 using the example of /swɔp/, a word that has an inconsistently spelled rime.

Figure 1

An interactive-activation framework incorporating sublexical orthographic and phonological systems, along with a lemma level that captures lexical information. A phonetic input system feeds information to the phonological system, with no commitment made here as to its structure.

Explaining the consistency effect

As seen in the figure, there are sublexical representations of both phonological and orthographic form, each hierarchically organized in terms of grain-size. Lexical representation is found at a level beyond the form-based systems, namely, at a “lemma” level that mediates between form and meaning (see e.g., Levelt et al., 1999; Taft and Nguyen-Hoan, 2010). When an utterance is to be recognized, an attempt is made to activate a lexical representation (i.e., lemma) via the surface phonetic form (whose structure is left unspecified in the figure, as indicated by the question marks). However, because of the variability of phonetic input from speaker to speaker, it is hard to use the phonetic system to go directly to the lemma level. Instead, mediation is required through a representation that remains constant regardless of the speaker, and this is the role of the phonological system. The orthographic system can also provide supportive input to the lemma level and is automatically activated to do just that. Thus, interactive-activation is not contained within a single hierarchical system, but across more than one system where sublexical information is represented in different forms.

Within this framework, phonological units send activation directly to orthographic units of the same grain-size, and the degree of such activation varies with the frequency of association between those units in the language. So, the phonological rime unit /ɔp/ is linked to two possible orthographic rime units OP and AP (referred to as “bodies,” see e.g., Taft, 1991), though the former is a much more common association (as indicated by the line having greater weight). Similarly, the phoneme /ɔ/ is linked to both O and A with a weighting that reflects their relative frequencies. As a result of these multiple links, there will be considerable competition within the system when an orthographically inconsistent word is presented such as [swɔp]. Along with input from the onset /sw/ (not depicted in the figure), the correct lemma will eventually be activated, but it will take more time than when there is no sublexical competition, as in the case of a consistent word like [stIŋ].

In relation to pseudowords (e.g., [twɔp]), inconsistently spelled rimes will still generate competition between the sublexical orthographic units. However, such competition might not have much impact on lexical decision responses given that it does not actually matter which units are activated, especially when there is one dominant spelling (such as /ɔp/ being spelled OP). Whether or not a consistency effect is observed with pseudowords would really depend on how much weight is placed on the partial activation happening at the lemma level. If a “no” response is only made once all partially activated lemmas have been rejected, competition between body units could delay the resolution of this activation. This would be compatible with the idea that a larger lexicality effect is associated with a greater consistency effect (see the discussion earlier in relation to the timing-out mechanism). On the other hand, it might be argued that conflict at the sublexical level reduces the amount of activation at the lemma level, which would work against finding a consistency effect. It can therefore be seen that the interpretation, and indeed the existence, of a consistency effect with pseudowords is still very much unresolved.

Explaining pseudohomograph priming

The pseudohomograph priming effect (Taft et al., 2008) is readily explained within the framework of Figure 1 even though the primes are pseudowords. In particular, the stimulus [swæp] will activate the lemma for swap via the rime–body link /æp/→AP. Thus, the appropriate lemma will be pre-activated when the target to be recognized is [swɔp]. In contrast, the lemma for stop will not be activated when the prime [stæp] is presented because the rime /æp/ is not linked to the body OP.

Explaining phoneme goodness ratings

There are several ways in which the phoneme goodness rating task of Cutler et al. (2009) might be influenced by orthography within this framework. First, the “goodness” of a phoneme might be determined by how easy it is to activate a lexical representation (i.e., a lemma) from the utterance in which it is contained. While the rimes /εs/ and /oIs/ are consistently spelled (i.e., ESS and OICE respectively), there is inconsistency at the phoneme–grapheme level. That is, the /s/ phonological unit will be linked with orthographic units for both S (or SS) and C, with the former being a far more frequent association than the latter. Therefore, at this level of lowest grain-size, the word [voIs] will experience greater competition than will [blεs]. This means that the lemma for voice will take longer to activate than the lemma for bless leading to lower phoneme goodness ratings. In contrast, pseudowords like [plεs] and [floIs] will fail to show any difference in ratings because no lemma is being fully activated.

A second possible explanation is that the phoneme goodness rating is based on how strongly the phoneme representation for /s/ is activated. With activation passing from phonology-to-orthography and back to phonology commensurate with the strength of the particular association, activation of the grapheme unit S (or SS) will feed back stronger activation to the phoneme unit /s/ than will the grapheme unit C. When the utterance is the word [blεs], activation at the lemma level will propagate down to the grapheme unit S and, therefore, provide support for the phoneme unit /s/. The grapheme unit S will not be activated via the lemma level when the utterance is the word [voIs]. With pseudowords (e.g., [plεs] or [floIs]), the body level might send activation to the grapheme level (e.g., /εs/→ESS→S), but to a much lesser extent than when there is a lexical contribution to the propagation of such activation. In this way, the grapheme unit S will provide very little support for the phoneme unit /s/ and therefore have negligible impact on phoneme goodness ratings.

Note that this account incorporates the involvement of a rebound mechanism from phonology-to-orthography and back to phonology (at the grapheme–phoneme level). Whether such second order feedback occurs between orthography and phonology in auditory processing was tested by Ziegler et al. (2008) with inconclusive results. Rime–body consistency showed the typically reported significant effect on the RT measure, with no effect for body–rime consistency. However, there was no interaction between the two. On the accuracy measure, there was a significant interaction between the two, but it was actually body–rime inconsistency that showed the interference, even though this was only a trend. Therefore, there was some evidence for orthographic-to-phonological activation in auditory processing, though maybe it was weakened by variability arising from the use of a lexical decision task that requires the comparison of different utterances in the different conditions. In contrast, the comparison of responses in the phoneme goodness rating task was made between variations of the same utterance, thus avoiding any less-than-perfect matching of items between conditions. In addition, although the phoneme rating task could be potentially performed at the lemma level (see the first explanation given above), it makes sense for a decision about the quality of a phoneme to be made at the phoneme level, hence involving a rebound from any orthographic units that might have been activated. On the other hand, lexical decision within the framework of Figure 1 must be based on activation at the lemma level, for which rebound from orthography back to phonology is not necessary. As such, evidence for second order feedback might be hard to observe in the auditory lexical decision task (or indeed the visual lexical decision task, see e.g., Peereman et al., 1998).

Orthographically Influenced Phonological Representations

Instead of assuming that the orthographic effects arise explicitly from representations of orthographic form, it is possible that orthographic knowledge modifies the nature of the mentally stored phonological information, and that the influence of orthography on spoken word processing arises indirectly from this phonological representation (e.g., Taft and Hambly, 1985; Peereman et al., 2009; Perre et al., 2009). A conceptualization of how orthography might be incorporated into the phonological system is given by Taft (2006b), who proposes that there is an abstract phonological level of representation that directly reflects the pronunciation of the spelling of the word (i.e., an orthographically influenced phonological representation or “OIP”). For example, because of its spelling, the word corn is represented at the OIP level with a post-vocalic r (i.e., /korn/) even for non-rhotic speakers of English who pronounce it as [kɔ:n]. Support for such an idea comes from the fact that it is hard for non-rhotic English speakers to recognize the homophony of a pseudohomophone that conflicts with its base word in terms of a post-vocalic r (Taft, 2006). For example, many non-rhotic speakers fail to recognize that cawn is homophonic with corn until they read it aloud. This implies that the representation of or and aw do not match in the underlying phonological system, and only coincide at the surface phonetic level.

Following this line of argument, the word /swɔp/ would be represented at the OIP level as /swæp/, reflecting the spelling swap (see Figure 2). The surface pronunciation /swɔp/ would then be generated at articulation from this underlying representation via a rule that converts /æ/ into /ɔ/ after /w/, perhaps when the /æ/ is especially marked. It is also possible that homophonic consonants are differentiated within these OIP representations as well, such as f and ph, or s and soft c. For example, ph might be represented as a variant of p (e.g., /ph/) that is transformed into /f/ at articulation, and soft c as a variant of c (e.g., /cs/) that is transformed into /s/ at articulation.

Figure 2

An interactive-activation framework incorporating orthographically influenced phonological (OIP) representations that are only available to output, phonological representations used as input to the lemma level, and a representation of surface phonetic form.

According to the account illustrated in Figure 2, there are two phonological systems, one corresponding to a phonemic version of the phonetic input and the other corresponding to the phonemic version of the orthography of the word. To provide a rationale for having two phonological systems, it is suggested that the former is available for input to the lemma level while the latter is an output system only. This is depicted in Figure 2 in terms of the direction of the arrows.

The explanations given for the various effects in terms of OIP differ from those framed in terms of orthography (i.e., Figure 1) primarily because the former is an output system and does not feed activation to the lemma level. The consistency effect arises from competition between the different OIP representations of the rime (e.g., /ɔp/ and /æp/) that are activated either directly from a rime unit in the input phonological system (/ɔp/) or via partially activated lemmas (e.g., for stop and swap). The greater the competition, the weaker the feedback to the lemma level via the input system. Pseudowords should also encounter competition, but unlike the case of real words, the lexical decision response does not require the activation of a single lemma to reach the threshold for recognition.

Pseudohomograph priming is explained in terms of pre-activation of the target lemma via the OIP rime unit. For example, the lemma for swap will be activated by the prime [swæp] when the input unit /æp/ (not depicted in Figure 2) activates the output unit /æp/, which in turn feeds activation back to the input unit /ɔp/.

The explanation for phoneme goodness ratings is similar to that put forward in relation to Figure 1. Instead of greater support for the phoneme /s/ coming from the orthographic representation S relative to C, the greater support in Figure 2 comes from the output phonological representation /s/ relative to the output phonological representation /cs/. The possible explanations given for the failure to find an effect with pseudowords remains the same as before, but framed in terms of output phonology instead of orthography.

It is hard to tease apart the direct involvement of orthography from its indirect involvement through orthographically influenced phonology, since they make very similar predictions in terms of behavioral data. However, Perre et al. (2009) favor the latter approach on the grounds that ERP data suggest that the cortical regions responsible for the consistency effect are those associated with phonological processing, and not those associated with visual word form.

Conclusion

The focus of the analysis given here of orthographic effects in spoken word recognition has been the impact on pseudowords. Consideration is given to a range of possible explanations for previously reported findings. Questions are raised about the validity of the findings in relation to the rime consistency effect with pseudowords (e.g., Ziegler and Ferrand, 1998; Ventura et al., 2004; Pattamadilok et al., 2007, 2009), but ways in which the presence or absence of such an effect might be handled are explored. Cutler et al. (2009) argue that their failure to find an orthographic influence for pseudowords on phoneme goodness ratings indicates a lack of orthographic influence in judgments about speech at the sublexical level. However, it is suggested here that their findings can be handled even when orthography is automatically activated sublexically. Finally, it is shown that the existence of a pseudohomograph effect (Taft et al., 2008) can be explained within the same framework as the other findings reported with pseudowords. While there is still much speculation associated with the explanations given here for orthographic effects in speech processing, the aim has been to provide an account of the most obvious options. Whether these can be teased apart empirically remains to be seen.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.