Face-Processing Differences Present in Grapheme-Color Synesthetes.

Grapheme-color synesthesia is a heterogeneous neurological phenomenon whereby the experience of a grapheme automatically and involuntarily elicits an experience of color. While the majority of synesthesia research has focused on inducer-specific influences of synesthetic associations, more recent efforts have examined potential broader differences. Based on spontaneous reports from synesthetes detailing problems with face recognition, in conjunction with the geographical proximity of neurological regions relevant to both synesthesia and face processing, we sought to examine whether synesthetes demonstrated atypical face-processing abilities. A total of 16 grapheme-color synesthetes and 16 age-and-gender matched controls (±3 years) completed the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006) of face memory, the Vanderbilt Holistic Face Processing Task (VHPT-F; Richler, Floyd, & Gauthier, 2014) of holistic face processing, as well as a standardized self-report questionnaire the Faces and Emotions Questionnaire (Freeman, Palermo, & Brock, 2015). The results revealed significantly poorer performance in synesthete's ability to recognize faces in the CFMT that was driven by a reduction in upright advantage. Results also revealed a significant reduction in overall accuracy on the VHPT-F for synesthetes, who despite this displayed a comparable holistic processing advantage compared to matched controls. Finally, synesthetes also rated themselves as significantly worse at face recognition. We suggest that this pattern may reflect differences in the development of individualized perceptual strategies.

Introduction

Grapheme‐color synesthesia is a heterogeneous neurological phenomenon whereby the experience of a grapheme automatically and involuntarily elicits an experience of color. An area that has received little attention in synesthesia research is cohabiting functions in the neural regions surrounding those known to be active in synesthetic experience. Previously, an explorative investigation by Sørensen (2013) reported face‐processing differences present in grapheme‐color synesthetes using a self‐report questionnaire, the Faces and Emotions Questionnaire (FEQ; Freeman, Palermo, & Brock, 2015) and the Cambridge Face Perception Test (CFPT; Duchaine, Germine, & Nakayama, 2007). He suggests that the proximity of primary regions known to be involved in synesthetic experiences to areas known to be selective for faces may offer some indication as to why some grapheme‐color synesthetes may experience face‐processing differences. The potential for such differences has further come to our attention via two independent and spontaneous reports made to us by synesthetes, where the common theme was problems with face recognition.

There are a number of key areas of atypical activation evidenced during the synesthetic experience, primarily in the fusiform gyrus (Hubbard, Arman, Ramachandran, & Boynton, 2005; Sperling, Pruvlovic, Linden, Singer, & Stirn, 2006). Synesthetes demonstrate increased activation in this area, speculated to be perhaps due to cross‐activation and differences in early neuronal pruning (see Brang, Hubbard, Coulson, Huang, & Ramachandran, 2010 for a more detailed explanation). More specifically, increased activation between areas V4, a primary visual area active during color perception (McKeefry & Zeki, 1997), and the visual word form area (VWFA), a region evidenced to show selective responses to orthographic stimuli (McCandliss, Cohen, & Dehaene, 2003; Moore, Durisko, Perfetti, & Fiez, 2014), have been reported in grapheme‐color synesthetes (Amsel, Kutas, & Coulson, 2017). The VWFA is also reported to be connected to another region that demonstrates relatively selective responses to number representations, the visual number form area (VNFA; Dehaene & Dehaene Lambertz, 2016). Located directly adjacent to the VWFA lies the fusiform expertise area, an area evidenced to selectively respond to categorized items of expertise. These include faces (Curby, Glazek, & Gauthier, 2009; Gauthier, Skudlarski, Gore, & Anderson, 2000; McGugin, Newton, Gore, & Gauthier, 2014) as part of the more general process of face recognition which relies on the projections from early visual areas to the occipital face area, fusiform gyrus, and superior temporal sulcus (e.g., Lohse et al., 2016).

Written words are also evidenced to interact with face perception. It is well established that the hemispheric specialization for faces alters from left to right during written language acquisition (Dundas, Plaut, & Behrmann, 2013). Further, it has been demonstrated that people with developmental dyslexia also have problems with face memory tasks (Sigurdardottir, Ívarsson, Kristinsdóttir, & Kristjánsson, 2015). Other more recent evidence also suggests that word processing and face processing are potentially reliant on broader shared processes. In a meta‐analysis of investigations looking at whether reading impairments are present in those with developmental prosopagnosia, Burns and Bukach (2021) report that poorer face memory was correlated with slower reading speeds and impoverished face perception is related to word lengths effects. Additionally, analysis of data from individuals with acquired prosopagnosia revealed that severe impairments in face recognition coincided with poorer word processing. The authors suggest that these data imply a shared relationship between face‐processing abilities and reading skills, which have been overlooked in previous studies that have failed to find correlations (e.g., Starrfelt, Klargaard, Petersen, & Gerlach, 2018).

Evidence of a close relation between color and faces can also be found in cases of patients who have suffered a brain injury (Bouvier & Engel, 2006). These regions share a common function in expertise, be it orthographic symbols or faces, and in the case of grapheme‐color synesthesia, increased activation in inducer‐concurrent selective areas not seen in neurotypical individuals. The geographical proximity of the VWFA, V4, and VNFA to the fusiform expertise and face area, coupled with recent suggestions that word and face‐processing are reliant on commonly shared processes (Burns & Bukach, 2021), and the findings of previous exploratory investigations (Sørensen, 2013) may support a hypothesis that atypical activation is not limited to those areas active during the synesthetic experience, but extends to other similar functions cohabiting the same region, resulting in individual differences in the perceptual strategies used for different object classes (Brogaard & Sørensen, in press a; Sørensen, 2019).

There has thus far been only one other investigation directly examining face‐processing abilities in grapheme‐color synesthesia (Janik McErlean, Susilo, Rezlescu, Bray, & Banissy, 2016), in addition to a more recent investigation indirectly examining face processing in synesthetes by utilization of an upright/inverted face recognition task to examine perceptual narrowing in synesthesia (Maurer et al., 2020). In their direct investigation, Janik McErlean et al. (2016) examined synesthetes’ abilities in face perception (using the CFPT; Duchaine et al., 2007), emotion perception abilities (CFPT‐angry, CFPT‐happy; Janik, Rezlescu, & Banissy, 2015), and holistic processing using a composite face‐processing task adapted from a previous investigation (Susilo, Rezlescu, & Duchaine, 2013). Their findings revealed that grapheme‐color synesthetes outperformed controls on tasks involving local processing of face identity and emotion but were comparable on tests of holistic face processing. The relevant experiments of the second investigation by Maurer et al. (2020) compared a mixed sample of self‐reported synesthetes and matched controls on recognition of upright and inverted human and chimpanzee faces. The human face stimuli consisted of a Caucasian female, and eight morphed “sister” images whose internal features were adjusted using Photoshop. The task required participants to match a target face to one of two faces that only differed in the spacing of internal facial features. Aggregated data from three experiments revealed synesthetes as better at overall face recognition; however, post hoc analyses found this effect to be driven by significant differences in the inverted task only. Synesthetes were better at matching inverted faces but were comparable with controls on upright faces.

Contrary to Sørensen (2013), the outcomes of both of these publications suggest that synesthetes may not have deficits in face processing. However, it is important to consider that the experiments presented in the recent publication from Maurer et al. (2020) were designed to examine local processing in synesthetes with varied forms of synesthesia and not specifically to test the face‐processing abilities of grapheme‐color synesthetes. Validation of tests designed to examine face processing is essential, especially considering the ongoing debates regarding the efficacy of relevant tests (e.g., Richler, Floyd, & Gauthier, 2014). Further, while not reporting a deficit per se, the results would suggest synesthetes possess a smaller upright advantage compared to controls, like the results from Sørensen (2013).

The results reported by Janik McErlean et al. (2016) contradict findings from a previous exploratory investigation. Here Sørensen (2013) showed that grapheme‐color synesthetes performed worse compared with non‐synesthetic controls, but still better than people with developmental prosopagnosia using the CFPT. Furthermore their responses on a self‐report task, the FEQ (Freeman et al., 2015) demonstrated an interaction where synesthetes appeared to compensate for the decreased performance in face recognition by being better at reading non‐facial emotional cues (e.g., the emotional tone of voice).

In recent years, the CFPT has become a less favored method for measuring face recognition abilities (Klargaard, 2019), which combined with the divergent results of the CFPT by Janik McErlean et al. (2016) and Sørensen (2013) raises questions as to the validity of this test in comprehensively determining face‐processing differences. Moreover, recent evidence from investigations into developmental prosopagnosia (Dalrymple, Fletcher, et al., 2014; Weigelt et al., 2014) implies that the tests used in the Janik McErlean et al. (2016) study may not be comprehensive in determining face‐processing differences. Face recognition models have suggested that face processing is in fact comprised of discrete independent phases (Bruce & Young, 1986), including processes that permit the representation of a face and its properties, termed face perception, and face memory which pertains to the processes that allow us to store, retain, and retrieve that information. Evidence of these being dissociable can be found in investigations into prosopagnosia, a neurocognitive disorder characterized by impaired face recognition (Bodamer, 1947; McConachie, 1976). While face memory is impaired in both adults and children with prosopagnosia, not all adult cases have displayed impairments in face perception (Busigny et al., 2014; Dalrymple, Fletcher, et al., 2014; Dalrymple, Garrido, & Duchaine, 2014). While there is not sufficient evidence at this time to suggest that potential face‐processing differences in synesthetes are limited to a specific type of face process, it is important at this stage of exploration to consider all tests available.

The evidenced correlations between face and word processing (Burns & Bukach, 2021), interactions, and subsequent changes to face‐selective areas during the acquisition of literacy (Dundas et al., 2013), in conjunction with the suspected areas of atypical cortical structure in close proximity to these areas have led researchers to explore whether broader categories that use high visual acuity are affected by synesthesia. These investigations are conflicted on whether differences in face processing are present in synesthetes, or are limited in their conclusions by the types of tests used. Additionally, more stringently validated methods of investigating holistic processing are now available (e.g., Richler et al., 2014). On the basis of these lingering issues, in conjunction with anecdotal reports from synesthetes detailing difficulties in face recognition, we further explore potential differences in face processing in grapheme‐color synesthetes by using standardized tests of face memory (Duchaine & Nakayama, 2006), holistic face processing (Richler et al., 2014), and a self‐report questionnaire (Freeman et al., 2015).

Method

Participants

Synesthetes

Sixteen grapheme‐color synesthetes (12 female, M = 27.9 years, SD = 8.07; four male, M = 32 years, SD = 17.9) were recruited via SynQuest an online self‐report questionnaire (https://www.communication.aau.dk/research/knowledge_groups/cnn/synesthesia/resources/synquest/) where they reported having grapheme‐color synesthesia. Synesthesia was verified via a consistency screening of their color associations using an in‐lab color selection tool (see Ásgeirsson, Nordfang, & Sørensen, 2015, pp. 8–9). A consistency score (see Eagleman, Kagan, Nelson, Sagaram, & Sarma, 2007, equations 1 and 2) cutoff point of 1.43 was used, based on previously established discrimination calculations for RGB mean values (Rothen, Seth, Witzel, & Ward, 2013). Synesthetes completed the screening, including the color wheel in a separate session than the main experiment. The minimum time between synesthetes completing the screening and taking part in the main experiment was 3 h to negate any effects of fatigue. All synesthetes produced consistency scores below this threshold (M = 0.692, minimum = 0.39, maximum = 1.3). Of the synesthetes, one was identified as belonging to the projector subtype, and all others were identified as belonging to the associator subtype. One associator synesthete reported also having chromesthesia, while all others reported only experiencing the grapheme‐color variant. Finally, one synesthete belonging to the associator subtype spontaneously reported difficulty in face perception prior to testing.

Controls

Age (±3 years) and gender‐matched controls (N = 16) participated in the experiment. The control participants were recruited via online advertisements. It was a mandatory requirement for controls to have no synesthesia, which was stated in the advertisement and verbally confirmed prior to testing. All participants received DKK 150 (approximately $ 25) for their participation.

Tasks

Face memory

The experiment used the Cambridge Face Memory Test (CFMT) (see Duchaine & Nakayama, 2006, for a detailed description) as a measure of face memory. The CFMT was chosen because it is specifically designed to measure face memory processes, which have not been specifically examined previously. Further, inversion tests of face processing are argued to be the most valid in determining face perception difficulties (Rezlescu, Susilo, Wilmer, & Caramazza, 2017). The stimuli for the CFMT include six target faces, each a Caucasian male with a neutral expression and no visible hair or facial blemishes. The task is comprised of three segments, the first of which measures recognition of the same image by presenting participants with an image of a target face, then asking them to identify the same image from a choice of three. The second segment measures recognition of novel images by asking participants to recognize the target face from a different viewing angle, while the third measures recognition from a different viewing angle with added Gaussian noise.

The variant of the test used in this experiment included both an “upright” and “inverted” version, whereby the experimental protocol is exactly repeated but with all stimuli inverted, to allow for an examination of upright advantage in face recognition (e.g., de Gelder, Bachoud‐Lévi, & Degos, 1998).

Holistic face processing

For a measure of holistic face processing, the experiment used the Vanderbilt Holistic Face Processing Task (VHPT‐F; Richler et al., 2014). This test has been demonstrated to be more reliable than other tests of holistic processing by its manipulation of image size and face area (Richler et al., 2014). The VHPT‐F is modeled on classic composite tests, where the measure of holistic face‐processing ability is indexed by the inability of an individual to selectively attend to a specific part of a face. Participants are presented with a face where the target area (either the upper, middle or lower part of the face) is highlighted using a red box. Participants are then presented with a test where the previously viewed target face area has been supplanted onto either a novel face or the same face, alongside two distractor faces. The distractor faces include the composition of non‐target face parts from the target face (see Fig. 1). Unlike classic composite tasks (e.g., Young et al., 1987), the VHPT‐F employs a three‐alternative forced‐choice task rather than a same/different approach, which reduces the risk of by‐chance correct responses. A more detailed task description can be found in Richler et al. (2014).

Fig. 1

Sample stimuli of the VHPT‐F using the upper half of the face as the target area. In the congruent trial, the correct response is the upper half of the target face presented in combination with the corresponding bottom half. In the incongruent trial, the target upper half of the study face is presented with an incongruent bottom half. Other trials differed by using either the middle or lower half of the face as target areas. Reprinted with the authors permission (from Richler et al., 2014).

Questionnaire

As a measure of self‐reported face‐processing ability, we included the FEQ (Freeman et al., 2015). The FEQ was used here in order to replicate previous self‐report results in synesthetes (Sørensen, 2013) as well as to implement the third measure of face‐processing ability, as there is discourse over which tests are most accurate at determining face‐processing differences (e.g., Richler et al., 2014). The questionnaire presents participants with scenarios and asks them to respond to how well each describes them using a 4‐point scale. It comprises various scenarios regarding face recognition (29 items), how well one can decipher emotional state from facial expressions (15 items), and how well one can detect emotional state from voice cues (10 items). The questionnaire was made available in both Danish and English and all participants completed the questionnaire in their native language.

Protocol

The stimuli were presented on a 24‐inch LCD monitor with an aspect ratio of 4:3, connected via display port to a MacBook Pro Mid‐2015 computer. Participants were seated 50 cm away from the monitor. Participants began with the upright and inverted versions of the CMFT, the order of version was counterbalanced between participants. Participants then completed the FEQ self‐report questionnaire, and finally the VHPT‐F. The total testing time for control participants was 1.5 h. A total testing time for synesthetes was on average 2.5 h on account of their initial screening. Screening for synesthesia was conducted in a separate testing session to negate potential fatigue effects.

Results

The analysis was conducted using SPSS statistics 25.0.0.2, and effect sizes Cohens d were calculated using G*power 3.1 (Faul, Erdfelder, Buchner, & Lang, 2009). Where t‐tests are used, Cohen's d is reported as a measure of effect size. For analyses of variance (anovas), ηp 2 is reported.

Cambridge face memory test

Overall accuracy was calculated using mean percentage correct. A t‐test conducted to compare overall accuracy of the CFMT revealed that control participants performed significantly better (M = 0.74, SD = 0.095) than synesthetes (M = 0.66, SD = 0.095), t(30) = 2.521, p = .01, d = 0.88. Analysis of accuracy by upright and inverted condition revealed a significant difference between synesthetes and controls for the upright condition (t(30) = 5.105, p < .01, d = 1.91), but no group differences for the inverted condition (t(30) = 0.089, p = .92, d = 0.03), Synesthetes showed reduced differences between upright (M = 0.72, SD = 0.12) and inverted faces (M = 0.61, SD = 0.09) compared with controls, who demonstrated greater accuracy for upright faces (M = 0.88, SD = 0.09) and inverted faces (M = 0.62, SD = 0.14).

Upright advantage score

To calculate upright advantage performance for our participants, the data were converted to a single ratio value by dividing the overall score from the inverted condition of the CFMT by the overall score of the upright condition. The data were then calculated using the formula , allowing for an intuitive single value between 0 and 1 in which a score of 0 corresponds to no upright advantage, and thus equal ability in face recognition regardless of whether a face is presented upright or inverted. Scores above 0 indicate the degree of advantage in recognizing upright faces. The benefit of the upright advantage score is that each participant provides his or her own baseline negating some of the noise that arises due to individual differences in the tasks.

An independent samples t‐test was conducted to examine potential differences in upright advantage score between synesthetes and controls for the CFMT. Results of the analysis indicated a significant difference in upright advantage between synesthetes and controls, t(30) = 3.144, p = .004, d = 1.11, where controls displayed a significantly greater upright advantage score (M = 0.32, SD = 0.15) than synesthetes (M = 0.16, SD = 0.14) (see Fig. 2).

Fig. 2

Average upright advantage scores for synesthetes and controls. A score of 0 corresponds to no upright advantage, and thus equal performance on recognition of both upright and inverted faces. Synesthetes’ lower score reflects a greater disadvantage for recognizing upright faces compared to controls. Bars represent 1 SE.

Holistic processing

Accuracy for the VHPT‐F was calculated as mean percentage correct. A t‐test conducted to examine overall performance revealed a significant difference between groups (t(30) = 2.445, p = .02, d = 0.87). Control participants (M = 0 .65, SD = 0 .09) performed significantly better than synesthetes (M = 0.60, SD = 0.10) for overall accuracy.

A 2 (congruent/incongruent) × 2 (synesthete/control) repeated measures anova examining holistic processing in the VHPT‐F revealed a significant effect of congruency overall (F(1, 30) = 86.917, p < .01, ηp 2 = 0.74), but did not reveal any significant group × congruency differences (F(1, 30) = 0.409, p  = .53, ηp 2 = 0.13). Synesthetes performed better in the congruent condition (M = 0.66, 95% CI [0.69, 0.76]) compared with the incongruent condition (M = 0.54, 95% CI [0.51, 0.58]). Controls also performed better in the congruent condition (M = 0.72, 95% CI [0.69, 0.76]) compared with the incongruent condition (M = 0.59, 95% CI [0.55, 0.62]). While synesthetes’ performance was poorer overall, both synesthetes and controls demonstrated an advantage for holistic face processing (see Fig. 3).

Fig. 3

(a) Overall accuracy for synesthetes and controls for the VHPT‐F and (b) averaged accuracy scores by congruency for the VHPT‐F. All bars represent 1 SE.

Questionnaire

In order to give a clearer overview of how these results are represented in the general population, normative data for the questionnaire were collected from undergraduate students at Aalborg University (N = 322). These additional normative data were included in the analysis. FEQ responses from all groups were quantified based on the ratings 1–4 (4 = strongly agree, 3 = slightly agree, 2 = slightly disagree, 1 = strongly disagree). Responses to reverse questions were then adjusted to coincide with the rating scheme of positive responses and then averaged per participant for each of the three subcategories.

A repeated measure anova using the group as a between‐subjects factor was conducted. The anova compared self‐reported abilities in face recognition, facial emotion recognition, and voice emotion recognition between synesthetes, controls, and normative sample participants. The analysis revealed a significant group × condition interaction (F(4, 702) = 41.204, p < .01, ηp 2 = 0.19). Post hoc analysis with Tukey correction revealed synesthetes reported themselves as significantly worse at face recognition (M = 2.46, 95% CI [2.3, 2.6]) than both controls (M = 3.13, 95% CI [3.15, 3.47]) and normative participants (M = 3.25, 95% CI 3.[21, 3.28]).

There were no differences in self‐reported face emotion recognition between synesthetes (M = 3.15, 95% CI [2.98, 3.31]), controls (M = 3.31, 95% CI [3.15, 3.48]), or norm participants (M = 3.16, 95% CI [3.13, 3.21]). Synesthetes rated themselves as significantly better at non‐facial emotional cues (M = 3.65, 95% CI [3.47, 3.83]) compared with both controls (M = 3.2, 95% CI [3.02, 3.37]) and norm participants (M = 2.94, 95% CI [2.91, 2.98]). Controls also rated themselves as better at reading non‐facial emotional cues compared with the norm sample (see Fig. 4).

Fig. 4

Averaged self‐report Likert ratings for face recognition, face emotion recognition, and vocal emotion recognition for synesthetes (n = 16), matched controls (n = 16), and normative sample (n = 322). Bars represent 1 SE. Differences between synesthetes and controls across the three measures demonstrated large effect sizes for face recognition (d = 3.8) and voice emotion (d = 1.5), and medium effect sizes for non‐significant face emotion comparison (d = 0.55).

Discussion

The current investigation sought to examine potential differences in the face‐processing abilities of grapheme‐color synesthetes. Analyses revealed significant between‐group differences, with synesthetes demonstrating poorer performance on the CFMT and a deficit in upright advantage in face recognition. Synesthetes also performed poorer in overall task accuracy for the VHPT‐F but demonstrated a comparable advantage for holistic processing. In addition, synesthetes reported their own abilities in face recognition as being poorer than controls, equivalent in face emotion recognition, and superior in detecting emotion from vocal cues. These results suggest that grapheme‐color synesthesia coincides with differences in face processing, potentially demonstrating a more far‐reaching influence of synesthesia over other perceptual processes. The results also indirectly support the suggestion by Burns and Bukach (2021) that word and face processing are not entirely dissociable.

Data presented here from the CFMT and FEQ further supports previous findings reported by Sørensen (2013) for the CFPT and FEQ by demonstrating an overall disadvantage in face processing for synesthetes. Although data presented by Janik McErlean et al. (2016) suggest no deficits in face‐processing abilities, the different tests used may be more sensitive to the differences of synesthetes. For example, it is possible that synesthetes’ differences are reflected in face memory processes only, as face perception and face memory have been shown to be dissociable in adults (Busigny et al., 2014), although further investigation is required to confirm whether specific aspects of face processing are affected. It is also important to note that both the CFPT and CFMT consist of relatively few trials (a total of eight in each condition) and may thus be more influenced by individual variation of participants, which may contribute to conflicting findings across experiments.

While overall accuracy in the VHPT‐F was reduced in synesthetes, they displayed comparable advantages for holistic processing as controls. This is in line with Janik McErlean et al. (2016), who found no group differences in holistic face processing using an alternative composite task. Previous investigations have reported a lack of correlation between composite effects and face recognition, and more specifically between outcomes of the VHPT and the CFMT (Richler, Floyd, & Gauthier, 2015). Well‐replicated effects in face processing including the composite effect, inversion effect, and part‐whole effect have been argued to actually represent distinct perceptual processes given their lack of correlation with face recognition and each other (Rezlescu et al., 2017). A recent comparison using a large sample (N = 282) found of these three components, the inversion effect had the highest correlation with overall face recognition abilities (Rezlescu et al., 2017). The data presented here indicate that while synesthetes show a reduction in face processing driven by a lack of upright advantage, this is not caused by a deficit in holistic face processing, supporting the theory that these are in fact separate processes.

We propose that differences in face perception ability in synesthetes, including an apparent lack of upright advantage, may be the result of the development of different perceptual strategies (Brogaard & Sørensen, in press a; Sørensen & Overgaard, 2018). Research has identified that grapheme‐color synesthesia is not apparent until the time an individual begins to become literate (Watson, Akins, Spiker, Crawford, & Enns, 2014) and may itself develop as a perceptual strategy in order to facilitate the acquisition of a new category (e.g., letters; Mannix & Sørensen, 2021). This key period of development coincides with the triggering of functional competition between words and faces. Words and faces experience joint hemispheric lateralization (Dundas et al., 2013), and a number of investigations have found concomitant effects on face processing during letter acquisition for both children and pre‐literate adults (Cantlon, Pinel, Dehaene, & Pelphrey, 2010; Dehaene et al., 2010). Others suggest that the neural tuning of faces in the left hemisphere is dependent on the neural tuning for letters and words, which are thought to become left lateralized due to their requirement for high visual acuity in discrimination and the convenience of proximity to language‐related areas (Dundas, Plaut, & Behrmann, 2014). Similarly, as face processing competes for representation in the mid‐fusiform region (Hasson, Levy, Behrmann, Hendler, & Malach, 2002), this would further promote hemispheric lateralization.

The dynamic lateralization of cognitive processing with increased environmental pressure may directly influence the development of individual differences. Specifically, differences in how individuals develop perceptual strategies (Brogaard & Sørensen, in press b; Sørensen & Overgaard, 2018), and more importantly which of these strategies consolidate over time can differ based on both the predisposition of the individual and interactions with the environment. It is well established that environmental factors shape the brain (e.g., Maguire, Woollett, & Spiers, 2006), and there is evidence that synesthesia too is influenced by external cues (Witthoft & Winawer, 2006; Witthoft, Winawer, & Eagleman, 2015). The development of the grapheme‐color variant of synesthesia, in conjunction with the dynamic activity occurring within its primary region of activation during development, may reflect differences in which perceptual strategies are adopted by synesthetes for visually complex object categories.

Similarly, it was recently demonstrated that perceptual processing in experts can be very similar to face processing (see also Curby et al., 2009). An exploration of the processing of Chinese characters and how familiarity and the number of feature elements (i.e., stroke count) influence perception found that processing is affected by familiarity independent of the number of feature elements (Dall, Wang, Cai, Chan, & Sørensen, 2021; see also Sun, Zimmer, & Fu, 2011). While perceptual discrimination is initially influenced by feature elements of an object, this reliance may change with increasing expertise (Popov & Reder, 2020, see Figure 8) to the degree that objects are processed in a holistic fashion independent of individual feature elements (Dall et al., 2021).

This suggests a high degree of perceptual variation based on environmental interaction and the consolidation of particular perceptual strategies (Brogaard & Sørensen, in press b). It also demonstrates that a gradual shift to holistic processing is not unique to faces but present in other areas of expertise, including conceptualized graphemes and logograms as in Dall et al. (2021). Synesthetes’ reduced ability in face recognition combined with the presence of holistic advantage indicates that they are not deficient in the typical development of expertise, but at the same time are not comparable to the neurotypical population in face recognition. This reflects the processing of individual graphemes—synesthesia is not correlated with any literacy disability (Basirat & Hupé, 2020), but they do not appear to process graphemes in the same way as the neurotypical population (Amsel et al., 2017). This may represent a general pattern of development of alternative perceptual strategies for processes with similar cognitive demands.

Within this framework, the data presented here may reflect grapheme‐color associations influencing the development of typical perceptual strategies, potentially by atypical modulation of the lateralization changes in literacy development, leading synesthetes to emphasize other features such as emotional or vocal cues to compensate (Sørensen, 2013).

The developmental stages at which face perception and face memory processes mature may provide an explanation for this phenomenon. Face perception matures early in development, in line with the development of perception of other objects, while face memory has a far longer trajectory spanning the first 10 years of life or more (Weigelt et al., 2014). As grapheme‐color synesthesia is not apparent up to the point of developing literacy, the timeline of longer‐term development of face memory and the typical age of learning to read would make face memory more vulnerable to influence by synesthetic development.

Based on our results, we suggest that the competition between words and faces occurs after an individual begins to acquire letters and words as graphemic concepts, a developmental period that coincides with synesthetic development and at which face perception processes have matured but face memory processes remain in development (Weigelt et al., 2014). The co‐development of synesthesia during the acquisition of literacy may affect the typical parity of competition for representation between words and faces in individuals with grapheme‐color synesthesia, resulting in alternative perceptual strategies becoming more favorable for identifying people.

Conflict of interest

The authors declare no conflicts of interest.

Supplementary material

Click here for additional data file.