| Literature DB >> 28600716 |
Yi-Chuan Chen1, Charles Spence2.
Abstract
The present study compared the time courses of the cross-modal semantic priming effects elicited by naturalistic sounds and spoken words on visual picture processing. Following an auditory prime, a picture (or blank frame) was briefly presented and then immediately masked. The participants had to judge whether or not a picture had been presented. Naturalistic sounds consistently elicited a cross-modal semantic priming effect on visual sensitivity (d') for pictures (higher d' in the congruent than in the incongruent condition) at the 350-ms rather than at the 1,000-ms stimulus onset asynchrony (SOA). Spoken words mainly elicited a cross-modal semantic priming effect at the 1,000-ms rather than at the 350-ms SOA, but this effect was modulated by the order of testing these two SOAs. It would therefore appear that visual picture processing can be rapidly primed by naturalistic sounds via cross-modal associations, and this effect is short lived. In contrast, spoken words prime visual picture processing over a wider range of prime-target intervals, though this effect was conditioned by the prior context.Entities:
Keywords: Audiovisual; Multisensory; Priming; Semantic; Sensitivity
Mesh:
Year: 2018 PMID: 28600716 PMCID: PMC5990551 DOI: 10.3758/s13423-017-1324-6
Source DB: PubMed Journal: Psychon Bull Rev ISSN: 1069-9384
Fig. 1a Sequence of three frames presented in each trial: A blank, a target picture (e.g., a dog), and a pattern mask. The target picture and pattern mask were presented in black in the center of a white background. b Represents the current experimental design in terms of signal detection theory. The distributions of dashed, dotted, and solid lines represent target present/congruent, target present/incongruent, and target absent conditions, respectively. In this design, the congruent and incongruent conditions share the same FA rate. The sensitivity (d') was calculated using the equations: d' = z(hit rate) – z(FA rate) in the congruent and incongruent conditions, separately (Green & Swets, 1966; Macmillan & Creelman, 2005)
Percentage of hit and false alarm (FA) rates (SE in parentheses) in each of the conditions in Experiment 1
| SOA (ms) | Sound type | Hit rate | FA rate | |
|---|---|---|---|---|
| Congruent | Incongruent | |||
| 350 | Naturalistic sound | 83.4 (3.8) | 79.1 (4.0) | 7.1 (2.6) |
| Spoken word | 84.0 (3.6) | 81.9 (4.6) | 6.7 (2.1) | |
| 1,000 | Naturalistic sound | 65.3 (3.7) | 68.3 (4.2) | 10.3 (4.0) |
| Spoken word | 73.8 (3.5) | 67.3 (3.9) | 12.4 (3.6) | |
Fig. 2Mean sensitivity (d') at the 350- and 1,000-ms SOAs in Experiment 1. Error bars indicate ±1 SEM. Sound = naturalistic sounds; Word = spoken words
Results of analysis of sensitivity (d') in Experiment 1 (three-way ANOVA: Congruency × Prime Type × SOA)
| Effect |
|
| ηp 2 | Note |
|---|---|---|---|---|
| Congruency | 5.92 | <.05 | 0.14 | Congruent (2.58) > Incongruent (2.50) |
| SOA | 11.22 | <.005 | 0.23 | 350 ms (2.99) > 1,000 ms (2.10) |
| Congruency × Prime Type × SOA | 9.99 | <.005 | 0.21 |
Fig. 3Mean sensitivity (d') at the 350- and 1,000-ms SOAs for Group 1 (tested in the order 350- and then 1,000-ms SOA) and Group 2 (with the order reversed) in Experiment 2. Error bars indicate ±1 SEM. Sound = naturalistic sounds; Word = spoken words
Percentage of hit and false alarm (FA) rates (SE in parentheses) in each of the conditions in Experiment 2
| SOA (ms) | Sound type | Hit rate | FA rate | ||
|---|---|---|---|---|---|
| Congruent | Incongruent | ||||
| Group 1 | 350 (first session) | Naturalistic sound | 88.0 (2.3) | 81.0 (4.3) | 3.2 (1.4) |
| Spoken word | 82.6 (3.8) | 81.4 (4.0) | 2.2 (0.6) | ||
| 1,000 (second session) | Naturalistic sound | 71.9 (3.7) | 72.3 (4.1) | 2.5 (0.9) | |
| Spoken word | 71.4 (4.4) | 69.0 (5.2) | 1.4 (0.4) | ||
| Group 2 | 350 (second session) | Naturalistic sound | 80.7 (4.0) | 75.8 (5.1) | 5.6 (3.3) |
| Spoken word | 79.2 (3.8) | 67.4 (5.4) | 3.5 (2.2) | ||
| 1,000 (first session) | Naturalistic sound | 69.4 (4.0) | 69.7 (4.5) | 4.7 (2.2) | |
| Spoken word | 71.1 (4.9) | 64.0 (5.9) | 5.5 (2.3) | ||
Results of analysis of sensitivity (d') in Experiment 2
| Effect |
|
| ηp 2 | Note |
|---|---|---|---|---|
| (A) Four-way ANOVA (Congruency × Prime Type × SOA × Order) | ||||
| Congruency | 17.85 | <.001 | 0.34 | Congruent (2.99) > Incongruent (2.85) |
| SOA | 45.32 | <.001 | 0.57 | 350 ms (3.13) > 1,000 ms (2.70) |
| Congruency × SOA | 10.15 | <.005 | 0.23 | |
| Congruency × Prime Type | 6.46 | <.05 | 0.16 | |
| Congruency × Order | 4.49 | <.05 | 0.12 | |
| Congruency × Prime Type × Order | 16.95 | <.001 | 0.33 | |
| (B) First session: three-way ANOVA (Congruency × Prime Type × SOA) | ||||
| Congruency | 8.85 | <.01 | 0.21 | Congruent (2.98) > Incongruent (2.85) |
| SOA | 6.65 | <.05 | 0.16 | 350 ms (3.26) > 1,000 ms (2.58) |
| Congruency × Prime Type × SOA | 12.24 | <.005 | 0.27 | |
| (C) Second session: three-way ANOVA (Congruency × Prime Type × SOA) | ||||
| Congruency | 15.83 | <.001 | 0.32 | Congruent (2.99) > Incongruent (2.84) |
| Congruency × SOA | 14.41 | <.005 | 0.30 | |
| Congruency × Prime Type | 4.96 | <.05 | 0.13 | |
| Naturalistic sounds | Spoken words | |||||||||
| Picture(Spoken word) | Naturalistic sound | Duration (ms) | Identification accuracy (%) | Mean confidence rating | Mean familiarity rating | Mean imagery concordance | Identification accuracy (%) | Mean confidence rating | Mean familiarity rating | Mean imagery concordance |
| Bird | Bird chirping | 350 | 77.8 | 4.7 | 4.8 | 4.4 | 100 | 6.9 | 6.9 | 4.7 |
| Cat | “Meow” | 350 | 100 | 6.2 | 6.3 | 4.4 | 100 | 7.0 | 7.0 | 4.8 |
| Cow | “Moo” | 350 | 88.9 | 6.4 | 6.3 | 4.8 | 100 | 6.9 | 7.0 | 4.8 |
| Dog | “Woof” | 350 | 94.4 | 6.4 | 6.4 | 4.3 | 100 | 6.9 | 6.9 | 4.8 |
| Duck | “Quack” | 350 | 100 | 6.8 | 6.3 | 4.5 | 100 | 6.9 | 6.9 | 4.8 |
| Eagle | Eagle call | 450 | 55.6 | 4.9 | 4.6 | 4.5 | 100 | 6.8 | 6.9 | 4.8 |
| Elephant | Roar | 500 | 61.1 | 4.8 | 4.6 | 4.7 | 100 | 6.9 | 6.9 | 4.9 |
| Frog | “Ribbit” | 350 | 83.3 | 5.1 | 5.1 | 4.2 | 100 | 7.0 | 7.0 | 4.9 |
| Goat | “Baa-baa” | 350 | 72.2 | 5.6 | 5.4 | 4.4 | 100 | 6.9 | 6.7 | 4.9 |
| Horse | Horse neighing | 350 | 61.1 | 5.2 | 4.9 | 4.0 | 94 | 6.2 | 6.7 | 4.9 |
| Pig | “Oink” | 350 | 50.0 | 5.2 | 5.4 | 4.2 | 100 | 6.9 | 7.0 | 4.9 |
| Rooster | Crowing sound | 450 | 88.9 | 5.8 | 5.6 | 4.7 | 100 | 6.9 | 6.2 | 4.8 |
| Car | Car starting | 350 | 77.8 | 5.4 | 5.8 | 4.0 | 100 | 7.0 | 7.0 | 4.6 |
| Door | Creak | 350 | 88.9 | 4.8 | 5.2 | 4.3 | 100 | 6.9 | 6.8 | 4.7 |
| Drum | Banging of a drum | 350 | 94.4 | 5.8 | 5.8 | 4.3 | 100 | 7.0 | 6.8 | 4.8 |
| Guitar | Guitar sound | 450 | 88.9 | 6.0 | 6.4 | 4.8 | 100 | 7.0 | 6.9 | 4.9 |
| Gun | Gunshot | 350 | 61.1 | 4.3 | 4.1 | 3.6 | 100 | 6.9 | 6.9 | 4.8 |
| Motorcycle | “Vroom-vroom” | 500 | 33.3 | 4.8 | 5.1 | 3.4 | 100 | 6.9 | 6.7 | 4.7 |
| Piano | Piano sound | 450 | 94.4 | 6.1 | 6.2 | 4.8 | 100 | 7.0 | 6.9 | 4.7 |
| Scissors | Clipping scissors | 450 | 22.2 | 3.2 | 3.8 | 3.7 | 100 | 6.9 | 6.9 | 4.9 |
| Switch | Click | 350 | 27.8 | 3.8 | 4.2 | 2.0 | 100 | 6.6 | 6.5 | 4.6 |
| Telephone | Telephone ringing | 500 | 100 | 6.6 | 6.7 | 4.7 | 100 | 7.0 | 7.0 | 4.6 |
| Trumpet | Trumpet sound | 450 | 83.3 | 5.3 | 5.3 | 4.6 | 100 | 7.0 | 6.7 | 4.9 |
| Whistle | Whistling | 450 | 66.7 | 5.4 | 5.5 | 4.8 | 94.4 | 6.8 | 6.6 | 4.9 |
| Mean ( | 60.9 (7.9) | 5.2 (0.2) | 5.2 (0.2) | 4.3 (0.1) | 98.7 (0.9) | 6.9 (0.1) | 6.8 (0.1) | 4.8 (0.02) | ||