Schedule of Events | Search Abstracts | Invited Symposia | Symposia | Poster Sessions | Data Blitz

Modeling human attentional priorities improves event boundary predictions in a computational model of event perception

Poster Session F - Tuesday, March 10, 2026, 8:00 – 10:00 am PDT, Fairview/Kitsilano Ballroom

Andrew Zhang¹ (), Wouter Kool², Jeffrey Zacks²; ¹Brown University, ²Washington University in St. Louis

Humans naturally form representations of ongoing events that help predict upcoming perceptual states. When prediction quality decreases, this may trigger updating of these representations–a process known as event segmentation. Forming and evaluating predictions depends importantly on attention, but current computational models do not well capture the human attentional priorities during event viewing. To address this limitation, we augmented the Structured Event Memory (SEM) model, (Nguyen et al., 2025, PNAS Nexus) with ground truth information about human attention using eye-tracked fixation data from naturalistic videos. We trained SEM with and without fixation information and assessed two measures central to event comprehension: detecting boundary points between events where prediction quality decreases (event segmentation) and forming event categories that support ongoing prediction and comprehension (event categorization). We quantified segmentation agreement as the alignment between each model’s predicted boundaries and human-marked boundaries, and categorization as the agreement between each model’s labeled event categories with human event categories. Attention weighting significantly improved agreement with human segmentation relative to the original model, while effects on categorization were inconsistent. These results highlight the importance of incorporating human attentional priorities in modeling event comprehension.

Topic Area: PERCEPTION & ACTION: Vision