Poster F125, Tuesday, March 27, 8:00-10:00 am, Exhibit Hall C
Perceptual decision making is supported by a hierarchical processing cascade, in both biological and artificial neural networks
Laura Gwilliams1,2, Jean-Rémi King3,1; 1New York University, 2NYU Abu Dhabi, 3Frankfurt Institute for Advanced Studies
Models of perceptual decision making have historically been designed to maximally explain behaviour and brain activity independently of their ability to actually perform tasks. More recently, performance-optimised models have been shown to correlate with brain responses to images and thus present a complementary approach to understand perceptual processes. In the present study, we compare how these approaches account for the spatio-temporal organisation of neural responses elicited by ambiguous visual stimuli. Forty-six healthy human subjects made perceptual decisions on briefly flashed stimuli constructed from ambiguous characters. The stimuli were designed to have 7 orthogonal properties, ranging from low-sensory levels (e.g. spatial location of the stimulus) to conceptual (whether stimulus is a letter or a digit) and task levels (i.e. required hand movement). Magneto-encephalography source and decoding analyses revealed that these 7 levels of representation are sequentially encoded by the cortical hierarchy, and actively maintained until the subject responds. This hierarchy appeared poorly correlated with a normative, drift-diffusion, and 5-layer convolutional neural network (CNN) optimised to accurately categorise alpha-numeric characters, but partially matched the sequence of activations of 3/6 state-of-the-art CNNs trained for natural image labeling (VGG-16, VGG-19, MobileNet). Additionally, we identify several systematic discrepancies between these neural networks and brain activity, revealing the importance of single-trial learning and recurrent processing. Overall, our results strengthen the notion that performance-optimised algorithms can converge towards the computational solution implemented by the human visual system, and open possible avenues to improve artificial perceptual decision making.
Topic Area: THINKING: Decision making