Spoken sentences decoding based on intracranial high gamma response using dynamic time warping

In this study, we explore the discriminability of high gamma activities from speech production cortex during the overt articulation of two sentences. Neural activities were recorded from one intracranial electrode placed approximately over the posterior part of the inferior frontal gyrus. By employing a dynamic time warping (DTW) method to realign single-trial high gamma response during speech productions, averaged temporal activation patterns corresponding to the two spoken sentences were obtained. Single-trial ECoG responses were subsequently classified according to their correlations with these two temporal activation patterns. On average, 77.5% of the trials were correctly classified, which was much higher than the chance-level performance of the SVM classifier without DTW. Our preliminary results shed light on the construction of cortical speech brain-computer interfaces on the sentence level.

[1]  Rajesh P. N. Rao,et al.  Cortical activity during motor execution, motor imagery, and imagery-based online feedback , 2010, Proceedings of the National Academy of Sciences.

[2]  Francois-Xavier Alario,et al.  Distinct representations of phonemes, syllables, and supra-syllabic sequences in the speech production network , 2009, NeuroImage.

[3]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[4]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[5]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[6]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[7]  Brian N. Pasley,et al.  Reconstructing Speech from Human Auditory Cortex , 2012, PLoS biology.

[8]  T. Gasser,et al.  Transformations towards the normal distribution of broad band spectral parameters of the EEG. , 1982, Electroencephalography and clinical neurophysiology.

[9]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[10]  Jonathan R. Wolpaw,et al.  Brain-computer interfaces (BCIs) for communication and control , 2007, Assets '07.

[11]  Rajesh P. N. Rao,et al.  Cortical electrode localization from X-rays and simple mapping for electrocorticographic research: The “Location on Cortex” (LOC) package for MATLAB , 2007, Journal of Neuroscience Methods.

[12]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[13]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[14]  Bradley Greger,et al.  Decoding spoken words using local field potentials recorded from the cortical surface , 2010, Journal of neural engineering.

[15]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.