Cognitive abilities predicting semantic tracking of speech

Poster Session B - Sunday, April 14, 2024, 8:00 – 10:00 am EDT, Sheraton Hall ABC

Matthew Modrusan¹, Jaimy Hannah¹, Bruno Mesquita¹, Ingrid Johnsrude¹; ¹University of Western Ontario

Previous work from our lab has shown that tests sensitive to fluid intelligence and working memory predict speech intelligibility in noise – that is, the ability to report words from sentences masked by multi-talker babble noise. Here, we examine whether these cognitive domains also predict speech comprehension, by studying their relation to semantic tracking of speech. Participants completed a matrix reasoning task (test of fluid intelligence) and a backward digit span task (test of working memory), followed by EEG recording (16 channel) during a sentence comprehension task. Participants listened to 144 sentences, half presented clearly, and half presented with multi-talker babble (+4 dB signal-to-noise ratio). Each spoken sentence (e.g., “The competition ended as a draw”) was followed by two text phrases that shared no words with the sentence (e.g. game was tied and match finished early). Participants chose the phrase that best semantically matched the sentence (i.e., game was tied). We plan to use semantic temporal response functions (sTRFs) as an index of semantic tracking. TRFs represent a linear mapping between the EEG signals and, in this case, semantic features of each spoken sentence derived from the GLoVE language model (Pennington et al., 2014). The magnitude of the TRF deflection indicates the degree to which semantic context is encoded in the brain. We anticipate that individuals scoring higher on our cognitive measures will show greater semantic tracking, especially for sentences with background noise. This would show that these cognitive abilities are important for naturalistic speech comprehension, beyond word report.

Topic Area: LANGUAGE: Semantic