Poster A56, Saturday, March 25, 5:00 – 7:00 pm, Pacific Concourse
Decoding Linguistic Structure Building in the Time-Frequency Domain
Phillip M. Alday1, Andrea E. Martin2,3; 1University of South Australia, 2Max Planck Institute for Psycholinguistics, 3University of Edinburgh
Linking hypotheses between cortical oscillations and the hierarchical structure of speech and language posit a correspondence across multiple timescales and levels of representation: fine speech structure is represented in the gamma band, while the speech envelope, i.e. syllables and words, in the alpha and theta bands (Ding et al., 2016; Giraud & Poeppel, 2012; Peelle & Davis, 2012). Detection of these signals is difficult because frequency-based encoding of stimulus-related information is distributed in cortical time and space. Addressing this problem in the fMRI signal, Multivariate Pattern Analysis (MVPA) extracts a stimulus-related abstract neural code (Haxby et al. 2011, 2014). Applied to M/EEG, MVPA yields enriched temporal information via the generalization across time method (GAT), which extracts and compares spatial patterns across time points (King & Dehaene, 2014). We tested the correspondence hypothesis by applying GAT to scalp EEG data from spoken pentasyllabic German words with either correct or incorrect lexical stress from Knaus (2013). An L1-regularized logistic regression classifier was trained and tested across syllable positions (time) on power-spectral density to predict correct stress. As predicted by Giraud & Poeppel (2012), alpha and theta activity displayed the strongest coupling with correct stress, and therefore, with lexical access, while beta and higher displayed almost none. A similarly trained classifier on time-domain (ERP) data performed poorly, indicating that lexical access relies crucially on information carried in alpha/theta oscillations. Our results offer a first quantitative estimation of the relative weightings of frequency bands carrying information required for lexical access during spoken word recognition.
Topic Area: LANGUAGE: Other