Literature DB >> 28966930

Tolerance for audiovisual asynchrony is enhanced by the spectrotemporal fidelity of the speaker's mouth movements and speech.

Antoine J Shahin1, Stanley Shen1, Jess R Kerlin1.   

Abstract

We examined the relationship between tolerance for audiovisual onset asynchrony (AVOA) and the spectrotemporal fidelity of the spoken words and the speaker's mouth movements. In two experiments that only varied in the temporal order of sensory modality, visual speech leading (exp1) or lagging (exp2) acoustic speech, participants watched intact and blurred videos of a speaker uttering trisyllabic words and nonwords that were noise vocoded with 4-, 8-, 16-, and 32-channels. They judged whether the speaker's mouth movements and the speech sounds were in-sync or out-of-sync. Individuals perceived synchrony (tolerated AVOA) on more trials when the acoustic speech was more speech-like (8 channels and higher vs. 4 channels), and when visual speech was intact than blurred (exp1 only). These findings suggest that enhanced spectrotemporal fidelity of the audiovisual (AV) signal prompts the brain to widen the window of integration promoting the fusion of temporally distant AV percepts.

Entities:  

Keywords:  Audiovisual integration; Audiovisual onset asynchrony; Degraded speech; Spectrotemporal fidelity

Year:  2017        PMID: 28966930      PMCID: PMC5617130          DOI: 10.1080/23273798.2017.1283428

Source DB:  PubMed          Journal:  Lang Cogn Neurosci        ISSN: 2327-3798            Impact factor:   2.331


  64 in total

1.  The functional neuroanatomy of language.

Authors:  Gregory Hickok
Journal:  Phys Life Rev       Date:  2009-09       Impact factor: 11.025

2.  Perceptual fusion and stimulus coincidence in the cross-modal integration of speech.

Authors:  Lee M Miller; Mark D'Esposito
Journal:  J Neurosci       Date:  2005-06-22       Impact factor: 6.167

3.  Audiovisual gating and the time course of speech perception.

Authors:  K G Munhall; Y Tohkura
Journal:  J Acoust Soc Am       Date:  1998-07       Impact factor: 1.840

4.  The continuity illusion does not depend on attentional state: FMRI evidence from illusory vowels.

Authors:  Antje Heinrich; Robert P Carlyon; Matthew H Davis; Ingrid S Johnsrude
Journal:  J Cogn Neurosci       Date:  2011-01-26       Impact factor: 3.225

5.  Functional specialization in rhesus monkey auditory cortex.

Authors:  B Tian; D Reser; A Durham; A Kustov; J P Rauschecker
Journal:  Science       Date:  2001-04-13       Impact factor: 47.728

6.  Visual speech acts differently than lexical context in supporting speech perception.

Authors:  Arthur G Samuel; Jerrold Lieblich
Journal:  J Exp Psychol Hum Percept Perform       Date:  2014-04-21       Impact factor: 3.332

7.  Auditory-visual speech perception and synchrony detection for speech and nonspeech signals.

Authors:  Brianna Conrey; David B Pisoni
Journal:  J Acoust Soc Am       Date:  2006-06       Impact factor: 1.840

8.  Speech recognition with primarily temporal cues.

Authors:  R V Shannon; F G Zeng; V Kamath; J Wygonski; M Ekelid
Journal:  Science       Date:  1995-10-13       Impact factor: 47.728

9.  Auditory and visual lexical neighborhoods in audiovisual speech perception.

Authors:  Nancy Tye-Murray; Mitchell Sommers; Brent Spehar
Journal:  Trends Amplif       Date:  2007-12

10.  Reconstructing speech from human auditory cortex.

Authors:  Brian N Pasley; Stephen V David; Nima Mesgarani; Adeen Flinker; Shihab A Shamma; Nathan E Crone; Robert T Knight; Edward F Chang
Journal:  PLoS Biol       Date:  2012-01-31       Impact factor: 8.029

View more
  3 in total

1.  Neurophysiology underlying influence of stimulus reliability on audiovisual integration.

Authors:  Hannah Shatzer; Stanley Shen; Jess R Kerlin; Mark A Pitt; Antoine J Shahin
Journal:  Eur J Neurosci       Date:  2018-02-09       Impact factor: 3.386

2.  Bayesian binding and fusion models explain illusion and enhancement effects in audiovisual speech perception.

Authors:  Alma Lindborg; Tobias S Andersen
Journal:  PLoS One       Date:  2021-02-19       Impact factor: 3.240

3.  Multisensory correlation computations in the human brain identified by a time-resolved encoding model.

Authors:  Jacques Pesnot Lerousseau; Cesare V Parise; Marc O Ernst; Virginie van Wassenhove
Journal:  Nat Commun       Date:  2022-05-05       Impact factor: 17.694

  3 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.