Discrete hierarchy of temporal receptive windows from a deep neural network using continuous time cells

Poster Session D - Monday, April 15, 2024, 8:00 – 10:00 am EDT, Sheraton Hall ABC

Aakash Sarkar¹ (aakash18@bu.edu), Marc Howard¹; ¹Boston University

A body of work shows a hierarchy of temporal receptive windows (TRWs) in the human cortex. Different brain regions respond to different kinds of information organized at different timescales. Time cells, observed in the hippocampus and cortex are believed to be important in a neural sense of time. Rather than firing in discrete clusters, time cells fire in smooth continuous sequences. To understand whether discrete temporal receptive windows are consistent with a smooth distribution of time constants, we trained SITHCon (Scale-invariant Temporal History Convolutional Network) to predict language-like sequences, with discrete transitions between symbols at multiple hierarchical levels, analogous to phonemes, words, and sentences. SITHCon is a deep network with layers of time cells tracking ‘what’ happened ‘when’; learnable weights change the meaning of `what’ from layer to layer. Each layer has a continuum of log-spaced time constants. We found evidence that the deep network containing of continuous time constants was able to extract information at discrete levels of the hierarchy. We saw different characteristic timescales to the autocorrelation function over layers of the network, despite the time cells in each layer having precisely the same distribution of time constants. As in TRW experiments, we permuted the input sequence at scales corresponding to each hierarchical level. We found that earlier layers are affected at small shuffling scales but not larger ones, while higher layers are impacted severely and especially at larger shuffling timescales. Our results indicate that networks with continuous time constants can exhibit a hierarchy of temporal receptive windows.

Topic Area: LANGUAGE: Other