Schedule of Events | Search Abstracts | Invited Symposia | Symposia | Poster Sessions | Data Blitz

Sketchpad Series 

Reinstatement of Complex Semantic Memory Representations for Visual Scenes Captured through Language Embeddings

Poster Session B - Sunday, March 8, 2026, 8:00 – 10:00 am PDT, Fairview/Kitsilano Ballroom
Also presenting in Data Blitz Session 1 - Saturday, March 7, 2026, 10:30 am – 12:00 pm PST, Salon ABC.

Frederik Bergmann¹, Roland Benoit¹; ¹University of Colorado at Boulder

When we recall complex scenes, we experience them with a wealth of semantic information. Here we used representational similarity analysis (RSA) and language encoders to test the hypothesis that this information is reinstated, in its semantic richness, during cued recall. We used the multilingual Universal Sentence Encoder (USE) to obtain embeddings of normative descriptions of complex visual scenes that participants were cued to remember while undergoing functional magnetic resonance imaging (fMRI). This allowed us to investigate whether activity patterns during recall correspond to semantic embedding structure. We focused on the anterior temporal (AT) and posterior medial (PM) networks and hypothesized that the latter should reflect the rich semantic information of complex scenes. Indeed, only the PM network showed significant alignment between neural and language-derived scene similarity, consistent with the PM’s established role in representing contextual-relational structure. A preliminary whole-brain searchlight RSA also revealed a bilateral region between the lateral occipital and temporal cortices. Critically, these effects were diminished or even absent when participants were given a cue but instructed not to retrieve the associated scene. On one hand, this highlights that the scenes themselves – and not their cues – gave rise to the similarity structure observed. On the other, it also shows that such reinstatement is under voluntary control. In sum, our findings show that high-level semantic embeddings derived from language can effectively model the neural reinstatement of complex visual memories.

Topic Area: LONG-TERM MEMORY: Episodic