Poster C63, Sunday, March 26, 5:00 – 7:00 pm, Pacific Concourse
Multilayer neural network modeling of speech envelope prediction errors
Jona Sassenhagen1, Benjamin Gagl1, Christian J. Fiebach1; 1University of Frankfurt
Understanding speech critically relies on top-down, predictive processing. This allows effective encoding of bottom-up sensory information like the speech envelope. We hypothesize that the cortical auditory perception system entrains to the speech envelope in order to predict the speech envelope in the future. Then, bottom-up signaling can focus on the unpredicted sensory events (i.e. prediction errors) only. Therefore we expect that brain regions showing entrainment to the speech envelope should also reflect speech envelope prediction errors. In a naturalistic language study, we modeled the raw audio envelope of the speech stream with a 5-layer recurrent neural network. We "pre-trained" this model on multiple hours of audio books, before allowing it to "entrain" on the actual stimuli and their particular characteristics. This model described the data better than models without “pre-training” (reflecting a benefit from general knowledge of speech) or without "entrainment" (reflecting a benefit from factoring in specifics of the speaker and the acoustic environment). Prediction errors are calculated by letting the models sequentially predict the (previously ‘unseen’) speech stimulus envelope and taking the absolute of the difference to the real envelope. These prediction errors are entered into a regression model to predict continuous MEG data from 14 subjects listening to the speech stimuli, controlling for both the actual speech envelope and its first derivative. We observed that prediction errors are reliably reflected by activity at Wernicke’s area around 55 ms. This indicates that neuronal signaling of speech signals is optimized by predictive processes, allowing neuronal efficient spoken language perception.
Topic Area: LANGUAGE: Other