An expected visual location biases observers' perception and neural encoding of sound locations

Poster Session A - Saturday, March 29, 2025, 3:00 – 5:00 pm EDT, Back Bay Ballroom/Republic Ballroom

Claire Pleche¹, Uta Noppeney¹; ¹Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands

To interact with the multisensory world the brain combines noisy sensory signals with prior knowledge into a coherent percept. While integrating information from multiple sources reduces uncertainty, it can introduce perceptual biases as illustrated by the ventriloquist illusion: When presented with spatially disparate audiovisual signals, observers perceive the sound as shifted towards the visual location. We investigated whether prior expectations about the location of a visual stimulus biases an observer’s perception and neural encoding of sound locations. Participants observed a visual object moving from various locations along the azimuth to the bottom of the screen, its trajectory partially occluded by a wall. When the object was expected to hit the ground, a sound was presented alone or in synchrony with the ball’s reappearance at the expected location. The sound came either from the ball’s: (i) expected location, (ii) final location before the occlusion or (iii) an unexpected location. Observers indicated their perceived sound location via a keypress. Participants’ perceived sound location was shifted towards the ball’s final location before the occlusion and also to the location where it was expected to reappear. Additionally, the neural response differed when the sound was presented at a spatially predicted compared to unpredicted location. Our results demonstrate that both a visual but also an expected visual input can bias where observers perceive sounds. From a Bayesian perspective, they show that the brain forms prior expectations based on inputs in one sensory modality that to some extent affect perceptual inference of stimuli in another modality.

Topic Area: PERCEPTION & ACTION: Multisensory