Formation of cognitive maps in AI agents through reinforcement learning using visual information: place cells and head-direction cells

Poster Session F - Tuesday, April 1, 2025, 8:00 – 10:00 am EDT, Back Bay Ballroom/Republic Ballroom

Aoto Kidachi¹ (g21230078e@edu.teu.ac.jp), Masayuki Kikuchi¹; ¹Computer Science Program, Department of Computer and Media Science, Graduate School of Bionics, Tokyo University of Technology

In animals, cognitive maps are formed by neurons in the hippocampus, such as place cells and head direction cells (HD cells). Recent studies show that AI agents in virtual environments can also form cognitive maps like those of animals. They have shown that such a characteristic of the AI agents are formed through a deep learning method called Long Short-Term Memory (which controls memory and forgetting to learn long-term connections). However, in most of those studies, AI agents have relied on data that real animals cannot directly perceive, such as their own coordinates, direction, and distances to objects. Therefore, the biological plausibility of the cognitive maps formed by these AI agents are questionable. Previously, the authors discovered that place cells emerge using Rainbow (a deep reinforcement learning method) that performs goal-search tasks based solely on visual information. This study shows that the spatial size of the place field (the region where place cells are active) correlated with environment size, and the findings are consistent with animal experiments. We applied the Rayleigh test (p < 0.05) to neurons in the visual processing layer (CNN output layer), the state value encoding layer (Value layer), and the action value encoding layer (Advantage layer) within Rainbow, confirming that HD cells were observed in every layer. These results show that the Rainbow model-a biologically plausible approach, trained exclusively on visual data rather than relying on artificial data unavailable to animals-can obtain neural representations of cognitive maps like those observed in animals.

Topic Area: PERCEPTION & ACTION: Other