Temporal information processing and memory guided behaviors with recurrent neural networks

Abstract

The ability to quantify temporal information on the scale of hundreds of milliseconds is critical towards the processing of complex sensory and motor patterns. However, the nature of neural mechanisms for temporal information processing (at this scale) in the brain still remains largely unknown. Furthermore, given that biological organisms are situated in a dynamic environment, the processing of time-varying environmental stimuli is intricately related to the generation of cognitive behaviors, and as such, an important element of learning and memory. In order to model such temporal processing recurrent neural networks emerge as natural candidates due to their inherent dynamics and fading memory of advent stimuli. As such, this thesis investigates recurrent neural network (RNN) models driven by external stimuli as the basis of time perception and temporal processing in the brain. Such processing lies in the short timescale that is responsible for the generation of short-term memory-guided behaviors like complex motor pattern processing and generation, motor prediction, time-delayed responses, and goal-directed decision making. We present a novel self-adaptive RNN model and verify its ability to generate such complex temporally dependent behaviors, juxtaposing it critically with current state of the art non-adaptive or static RNN models. Taking into consideration the brain's ability to undergo changes at structural and functional levels across a wide range of time spans, in this thesis, we make the primary hypothesis, that a combination of neuronal plasticity and homeostatic mechanisms in conjunction with the innate recurrent loops in the underlying neural circuitry gives rise to such temporally-guided actions. Furthermore, unlike most previous studies of spatio-temporal processing in the brain, here we follow a closed-loop approach. Such that, there is a tight coupling between the neural computations and the resultant behaviors, demonstrated on artificial robotic agents as the embodied self of a biological organism. In the first part of the thesis, using a RNN model of rate-coded neurons starting with random initialization of synaptic connections, we propose a learning rule based on local active information storage (LAIS). This is measured at each spatiotemporal location of the network, and used to adapt the individual neuronal decay rates or time constants with respect to the incoming stimuli. This allows an adaptive timescale of the network according to changes in timescales of inputs. We combine this, with a mathematically derived, generalized mutual information driven intrinsic plasticity mechanism that can tune the non-linearity of network neurons. This enables the network to maintain homeostasis as well as, maximize the flow of information from input stimuli to neuronal outputs. These unsupervised local adaptations are then combined with supervised synaptic plasticity in order to tune the otherwise fixed synaptic connections, in a task dependent manner. The resultant plastic network, significantly outperforms previous static models for complex temporal processing tasks in non-linear computing power, temporal memory capacity, noise robustness as well as tuning towards near-critical dynamics. These are displayed using a number of benchmark tests, delayed memory guided responses with a robotic agent in real environment and complex motor pattern generation tasks. Furthermore, we also demonstrate the ability of our adaptive network to generate clock like behaviors underlying time perception in the brain. The model output matches the linear relationship of variance and squared time interval as observed from experimental studies. In the second part of the thesis, we first demonstrate the application of our model on behaviorally relevant motor prediction tasks with a walking robot, implementing distributed internal forward models using our adaptive network. Following this, we extend the previous supervised learning scheme, by imp