Learning the Sensorimotor Structure of the Foveated Retina

Abstract

We identify two properties of the human vision system, the foveated retina, and the ability to saccade, and show how these two properties are sufficient to simultaneously learn the structure of receptive fields in the retina and a saccade policy that centers the fovea on points of interest in a scene. We consider a novel learning algorithm under this model, sensorimotor embedding, which we evaluate using a simulated roving eye robot on synthetic and natural scenes, and physical pan/tilt camera. In each case we compare learned geometry to actual geometry, as well as the learned motor policy to the optimal motor policy. In both the simulated roving eye experiments and the physical pan/tilt camera, our algorithm is able to learn both an approximate sensor map and an effective saccade policy. The developmental nature of sensorimotor embedding allows an agent to simultaneously adapt both geometry and policy to changes in the physical model and motor properties of the retina. We demonstrate adaption in the case of retinal lesioning and motor map reversal. 1.