Semantic novelty modulates neural responses to sensory stimuli across the human brain
Maximilian Nentwich1,2 (firstname.lastname@example.org), Marcin Leszczynski3,4,5, Brian E. Russ4,6,7, Lukas Hirsch1, Noah Markowitz2, Kaustubh Sapru1, Charles E. Schroeder3,4, Ashesh D. Mehta2,8, Stephan Bickel2,4,8, Lucas C. Parra1; 1The City College of New York, CUNY, New York, NY, 2The Feinstein Institutes of Medical Research, Northwell Health, Manhasset, NY, 3Columbia University College of Physicians and Surgeons, New York, NY, 4Nathan Kline Institute, Orangeburg, NY, 5Jagiellonian University, Kraków, Poland, 6Icahn School of Medicine, New York, NY, 7New York University at Langone, New York, NY, 8Zucker School of Medicine at Hofstra/Northwell, Manhasset, NY
Our continuous sensory experience in daily life is dominated by change. Previous research has focused on change due to stimulus motion, eye movements, unfolding events, or auditory edges in speech. However, in naturalistic environments, these stimuli interact with each other and with semantic novelty. We investigate the neural responses to these distinct sources of visual change during film viewing. In addition, we investigate the multimodal interactions between visual change across saccadic eye movements with sensory and semantic features of speech. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobes. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.
Topic Area: PERCEPTION & ACTION: Multisensory
CNS Account Login
April 13–16 | 2024