Decoding lexical and supralexical processes in American Sign Language comprehension

Poster Session A - Saturday, April 13, 2024, 2:30 – 4:30 pm EDT, Sheraton Hall ABC

Brennan Terhune-Cotter^1,2 (bterhunecotter@sdsu.edu), Karen Emmorey¹; ¹San Diego State University, ²University of California, San Diego

Neural research on sign language comprehension has implicated a widespread frontotemporal network in processing the linguistic structure of signed input, but neural differences between lexical and supralexical (i.e., sentence level) processing have not been clearly delineated. We scanned fifteen deaf American Sign Language (ASL) signers as they passively observed sign lists and ASL sentences. Half of the sign lists consisted of nouns and half were verbs, allowing us to compare activation for nouns versus verbs. The sentences were matched to the sign lists in the number of content words and their psycholinguistic properties (frequency, iconicity, and phonological neighborhood density). We conducted multivoxel pattern analyses (MVPAs) using a whole-brain searchlight approach and a Support Vector Machine (SVM) classifier to distinguish brain regions that differentiate between (1) sentences and word lists, and (2) nouns and verbs. We decoded the neural activity patterns for each subject, then performed a group-level analysis on the results of the single-subject MVPA analyses. Regions that discriminated between nouns and verbs included bilateral (but left lateralized) superior temporal cortex, left angular gyrus, and bilateral occipital cortex. In contrast, only small regions within bilateral inferior frontal gyri discriminated between sentences and sign lists. Our results suggest that lexical information is robustly represented in the sign language network, whereas the representation of combinatorial syntax/semantics is limited to a small pair of homologous regions in bilateral inferior frontal cortex. These results cohere with recent research suggesting that lexical, rather than supralexical, information is more robustly represented in the spoken language network.

Topic Area: LANGUAGE: Lexicon