Uncertainty-driven updating enables segmentation and categorization of naturalistic activity

Poster Session E - Monday, April 15, 2024, 2:30 – 4:30 pm EDT, Sheraton Hall ABC

Tan Nguyen¹ (n.tan@wustl.edu), Matthew Bezdek¹, Samuel Gershman², Aaron Bobick³, Todd Braver¹, Jeffrey Zacks¹; ¹Department of Psychological and Brain Sciences, Washington University in St. Louis, ²Department of Psychology and Center for Brain Science, Harvard University, ³Computer Science and Engineering, Washington University in St. Louis

Humans form sequences of event models—representations of the immediate situation—to predict how activity will unfold. Multiple mechanisms have been proposed for how the cognitive system determines when to segment the stream of behavior and switch from one active event model to another. Here, we use a large-scale naturalistic dataset to compare two gating mechanisms for model updating: prediction uncertainty and prediction error. We constructed a computational model combining a recurrent neural network for short-term dynamics with Bayesian inference over event types for event-to-event transitions. This architecture learns event schemas representing knowledge about event types and uses them, along with observed perceptual information, to construct a series of event models. This architecture was trained on one pass through an 18-hour corpus of naturalistic human activity. Another 3.5 hours of activities were used to test each variant for agreement with human segmentation and categorization. The architecture was able to learn to predict human activity, and it developed human-like segmentation and categorization. We then compared two variants of this architecture designed to better emulate human event segmentation: one transitioned when the active event schema produced high uncertainty in its prediction; the other transitioned when the active event schema produced a large prediction error. The variant that transitioned from one active event schema to another based on prediction uncertainty provided the closest match to human segmentation and forming human-like event categories—despite being given no feedback about segmentation or categorization. These results establish that event model transitioning based on prediction uncertainty can naturally reproduce two important features of human event comprehension.

Topic Area: PERCEPTION & ACTION: Vision