Schedule of Events | Search Abstracts | Invited Symposia | Symposia | Poster Sessions | Data Blitz

Reading the signs: Encoding models generalize from English to ASL in novice signers

Poster Session D - Monday, March 9, 2026, 8:00 – 10:00 am PDT, Fairview/Kitsilano Ballroom

Megan E. Hillis¹ (), David J. M. Kraemer¹; ¹Dartmouth College

As someone learns a new language, do they map new information onto a semantic space that overlaps with language they already know? Evidence suggests that early-stage learners rely heavily on their established language to scaffold new learning, but many prior studies only utilize small lists of single words. Large language models (LLMs) have provided new avenues to probe neural processing of complex semantic information embedded in realistic utterances. Given recent findings that LLMs trained in different languages can be aligned to a shared embedding space with respect to semantic meaning, we sought to determine whether neural representations of language similarly converge on a shared neural substrate in novice learners. We present data from 32 English-speaking, nonsigning participants who underwent fMRI scanning before and after a three-week introductory course in American Sign Language (ASL). We found that after (but not before) learning, a voxelwise encoding model trained on the mapping of LLM embeddings to neural responses to English was able to predict neural responses to an independent set of ASL sentences in several regions including bilateral posterior superior temporal sulcus (pSTS) and left superior temporal gyrus (STG). The pSTS in particular has been implicated in sign language processing when cognitive demand is high, as in novice learners. As well as demonstrating the ability of encoding models to learn semantic mappings that generalize across language and modality, these results shed light on the neural underpinnings of emerging comprehension skills in the first few weeks of exposure to signed language.

Topic Area: LANGUAGE: Semantic