Connectionist Environment Modelling in a Real Robot

摘要

This paper describes some experiments with an adaptive controller, based on multi-layer perceptrons, which tries to solve a simple reinforcement learning task for a real robot vehicle. One neural network (the ‘model’) is trained to predict how the robot's sensor readings will change if it performs a given action; another learns, with the aid of the model, to evaluate sensory states according to how close the robot is to receiving a reward when it experiences them. Two kinds of model which exploit context information were evaluated in robot runs, as well as a ‘flat’ (memoryless) model. The results confirm that backprop can provide the learning mechanism needed to solve simple adaptive control tasks, and point up some problems which will need to be addressed before it can help with more complicated skills.