Intersubject neural similarity and pattern reinstatement during recall are enhanced at meaningful moments during film viewing

Poster Session C - Sunday, April 14, 2024, 5:00 – 7:00 pm EDT, Sheraton Hall ABC

Aditya Upadhyayula¹ (aditya.usa8@gmail.com), Jeffrey M. Zacks¹, John M. Henderson^2,3, Zachariah M. Reagh¹; ¹Depratment of Psychological & Brain Sciences, Washington University in St. Louis, ²Center for Mind and Brain, University of California, Davis, ³Department of Psychology, University of California, Davis

How do we represent and remember complex experiences? Recent work on event cognition suggests that we parse and organize such experiences in a meaningful way. Asking people to communicate their experiences by summarizing a sequence of events can reveal this organization. Here, we introduce a storyboard paradigm to investigate event representations. Participants watched 50 video clips (approximately 1-2 minutes each) from an episode of a television show (BBC’s Sherlock, S1E1: A Study in Pink). They were then asked to create a storyboard that conveyed the underlying story by selecting individual frames within each clip. There were no restrictions on the number of frames participants could select. We found remarkably high agreement among participants in selecting individual frames. In a separate analysis of an open fMRI dataset containing both the encoding and recall of the same video clips, we found that within the Posterior Medial Cortex (PMC), the probability of a particular timepoint containing a storyboard frame was positively correlated with the intersubject neural pattern similarity. Further, encoding patterns weighted by their respective storyboard probabilities also significantly correlated with the recall activity patterns even after accounting for the unweighted encoding patterns. Finally, we compared storyboard moments to event boundaries and found that storyboards capture distinct information from event boundaries, both behaviorally and neurally. These results suggest that storyboard moments capture crucial information about how we represent complex experiences. Storyboards allow us to study meaningful moments underlying experiences and can be useful for investigating episodic memory at a finer scale.

Topic Area: LONG-TERM MEMORY: Episodic