OBJECTIVE: In recent years, brain-computer interface (BCI) systems have been investigated for their potential as a communication device to assist people with severe paralysis. Decoding speech sensorimotor cortex activity is a promising avenue for the generation of BCI control signals, but is complicated by variability in neural patterns, leading to suboptimal decoding. We investigated whether neural pattern variability associated with sound pronunciation can be explained by prior pronunciations and determined to what extent prior speech affects BCI decoding accuracy. APPROACH: Neural patterns in speech motor areas were evaluated with electrocorticography in five epilepsy patients, who performed a simple speech task that involved pronunciation of the /i/ sound, preceded by either silence, the /a/ sound or the /u/ sound. MAIN RESULTS: The neural pattern related to the /i/ sound depends on previous sounds and is therefore associated with multiple distinct sensorimotor patterns, which is likely to reflect differences in the movements towards this sound. We also show that these patterns still contain a commonality that is distinct from the other vowel sounds (/a/ and /u/). Classification accuracies for the decoding of different sounds do increase, however, when the multiple patterns for the /i/ sound are taken into account. Simply including multiple forms of the /i/ vowel in the training set for the creation of a single /i/ model performs as well as training individual models for each /i/ variation. SIGNIFICANCE: Our results are of interest for the development of BCIs that aim to decode speech sounds from the sensorimotor cortex, since they argue that a multitude of cortical activity patterns associated with speech movements can be reduced to a basis set of models which reflect meaningful language units (vowels), yet it is important to account for the variety of neural patterns associated with a single sound in the training process.
OBJECTIVE: In recent years, brain-computer interface (BCI) systems have been investigated for their potential as a communication device to assist people with severe paralysis. Decoding speech sensorimotor cortex activity is a promising avenue for the generation of BCI control signals, but is complicated by variability in neural patterns, leading to suboptimal decoding. We investigated whether neural pattern variability associated with sound pronunciation can be explained by prior pronunciations and determined to what extent prior speech affects BCI decoding accuracy. APPROACH: Neural patterns in speech motor areas were evaluated with electrocorticography in five epilepsy patients, who performed a simple speech task that involved pronunciation of the /i/ sound, preceded by either silence, the /a/ sound or the /u/ sound. MAIN RESULTS: The neural pattern related to the /i/ sound depends on previous sounds and is therefore associated with multiple distinct sensorimotor patterns, which is likely to reflect differences in the movements towards this sound. We also show that these patterns still contain a commonality that is distinct from the other vowel sounds (/a/ and /u/). Classification accuracies for the decoding of different sounds do increase, however, when the multiple patterns for the /i/ sound are taken into account. Simply including multiple forms of the /i/ vowel in the training set for the creation of a single /i/ model performs as well as training individual models for each /i/ variation. SIGNIFICANCE: Our results are of interest for the development of BCIs that aim to decode speech sounds from the sensorimotor cortex, since they argue that a multitude of cortical activity patterns associated with speech movements can be reduced to a basis set of models which reflect meaningful language units (vowels), yet it is important to account for the variety of neural patterns associated with a single sound in the training process.
Authors: Jonathan R Wolpaw; Niels Birbaumer; Dennis J McFarland; Gert Pfurtscheller; Theresa M Vaughan Journal: Clin Neurophysiol Date: 2002-06 Impact factor: 3.708
Authors: Dora Hermes; Jeroen C W Siero; Erik J Aarnoutse; Frans S S Leijten; Natalia Petridou; Nick F Ramsey Journal: J Neurosci Date: 2012-07-11 Impact factor: 6.167
Authors: Jeroen C W Siero; Dora Hermes; Hans Hoogduin; Peter R Luijten; Nick F Ramsey; Natalia Petridou Journal: Neuroimage Date: 2014-07-12 Impact factor: 6.556
Authors: Kai J Miller; Eric C Leuthardt; Gerwin Schalk; Rajesh P N Rao; Nicholas R Anderson; Daniel W Moran; John W Miller; Jeffrey G Ojemann Journal: J Neurosci Date: 2007-02-28 Impact factor: 6.167
Authors: Jonathan S Brumberg; E Joe Wright; Dinal S Andreasen; Frank H Guenther; Philip R Kennedy Journal: Front Neurosci Date: 2011-05-12 Impact factor: 4.677
Authors: Mariska J Vansteensel; Elmar G M Pels; Erik J Aarnoutse; Martin G Bleichner; Mariana P Branco; Timothy Denison; Zachary V Freudenburg; Peter Gosselaar; Sacha Leinders; Thomas H Ottens; Max A Van Den Boom; Peter C Van Rijen; Nick F Ramsey Journal: N Engl J Med Date: 2016-11-12 Impact factor: 91.245
Authors: Jane E Huggins; Christoph Guger; Erik Aarnoutse; Brendan Allison; Charles W Anderson; Steven Bedrick; Walter Besio; Ricardo Chavarriaga; Jennifer L Collinger; An H Do; Christian Herff; Matthias Hohmann; Michelle Kinsella; Kyuhwa Lee; Fabien Lotte; Gernot Müller-Putz; Anton Nijholt; Elmar Pels; Betts Peters; Felix Putze; Rüdiger Rupp; Gerwin Schalk; Stephanie Scott; Michael Tangermann; Paul Tubig; Thorsten Zander Journal: Brain Comput Interfaces (Abingdon) Date: 2019-12-10
Authors: Guy H Wilson; Sergey D Stavisky; Francis R Willett; Donald T Avansino; Jessica N Kelemen; Leigh R Hochberg; Jaimie M Henderson; Shaul Druckmann; Krishna V Shenoy Journal: J Neural Eng Date: 2020-11-25 Impact factor: 5.379