Poster B81, Sunday, March 26, 8:00 – 10:00 am, Pacific Concourse
Representational similarity in the brain and computational language processing: New clues about the neural encoding of word meaning.
Francesca Carota1,2,3,4, Hamed Nili2,5, Nikolaus Kriegeskorte2,3, Friedemann Pulvermüller1,2,4; 1Humboldt Universit ät zu Berlin, Germany, 2MRC Cognition and Brain Sciences Unit, Cambridge, UK, 3University of Cambridge, Downing Street, Cambridge, CB2 3EB United Kingdom, 4Freie Universität, Berlin, Germany, 5University of Oxford, Oxford, UK
Language comprehension engages a distributed network of fronto-temporal, parietal and sensorimotor regions, but it is still unclear how meaning of words and their semantic relationships are represented and processed within these regions and to which degrees lexico-semantic representations differ between regions and semantic types. We used fMRI and Representational Similarity Analysis, RSA, to relate word-elicited multi-voxel patterns to latent semantic similarity among categories of action (face-, arm-, leg-related verbs) and object (animal-, food, tool-related nouns) words, as assessed by distributional statistics performed on text corpora (Latent Semantic Analysis, LSA). In left inferior frontal (BA 44-45-47), left posterior middle temporal and left precentral cortex, the similarity structure of brain response patterns conformed to the semantic similarity among action-related verbs, as well as - across lexical semantic categories - between action verbs and tool-related nouns and, to a degree, between action verbs and food nouns, but not between action verbs and animal nouns. Instead, posterior inferior temporal cortex exhibited a reverse response pattern, which reflected the semantic similarity among object-related nouns, but not action-related words. These results show that semantic similarity among categories of is encoded by a range of cortical areas, including multimodal association (e.g., anterior inferior frontal, posterior middle temporal) and modality-preferential (premotor) cortex and that the representational geometries in these regions are partly dependent on semantic type, with semantic similarity among action-related words crossing lexical-semantic category boundaries. Furthermore, these findings suggest that distributional information about word co-occurrence is relevant to shape word representations in the brain.
Topic Area: LANGUAGE: Semantic