Poster B135, Sunday, March 26, 8:00 – 10:00 am, Pacific Concourse
Comparing computational, object and functional models of scene representation in the human brain
Iris I A Groen1, Michelle R Greene2, Christopher Baldassano3, Li Fei-Fei2, Diane M Beck4, Chris I Baker1; 1National Institutes of Mental Health, 2Stanford University, 3Princeton University, 4University of Illinois
Complex scene perception is characterized by the activation of multiple regions in posterior cortex. So far, these regions have been mostly interpreted as representing visual characteristics of scenes, such as the depicted environment (“a kitchen”), constituent objects (“an oven”), or spatial layout (“a closed space”). Recent behavioral evidence, however, suggests that the functions afforded by a scene (e.g. “could I prepare food here?”) play a central role in how scenes are understood (Greene et al., 2016, JEP:General). Here, we studied whether the brain represents scene functions using a model-based approach. Healthy volunteers (n=20) viewed images from 30 scene categories in an ultra-high-field 7T MRI scanner. Stimuli were carefully selected from a larger set of scenes characterized in terms of their visual properties (derived computationally using a deep neural network), object occurrence, and scene function (derived using separate behavioral experiments), such that each model predicted a maximally different pattern of brain responses. We found that the visual model best predicted fMRI responses in scene-selective regions, with additional but limited contribution from the functional and object models. A whole brain analysis confirmed a strong contribution of the visual model throughout high-level visual cortex. The greatest correspondence with the functional model was observed in parts of anterior ventral and parietal cortex, potentially overlapping with a network involved in memory retrieval. Overall, these results show that while visual properties clearly drive brain responses to complex scenes most strongly, understanding complex scenes may also engage larger-scale networks beyond those revealed by simple visual activation experiments.
Topic Area: PERCEPTION & ACTION: Vision