Emergent intelligence in a distributed adaptive control system

Abstract

A method is presented in this thesis whereby reinforcement learning can be incorporated into a behavioral based control system for mobile robotic applications. In many situations the skills and behaviors required by the robot for productive operation and survival are impractical or impossible to predetermine and embed within the robot's control systems. In this method developed in this research, such skills and behaviors are automatically learned through self-exploration and self-organization using an incremental dynamic programing technique. Conventional predetermined responses are replaced with the learned responses form the Adaptive Behavior Modules (ABMs) developed in this thesis. These ABMs are capable of the autonomous discovery and learning of useful skills through unaided interaction within the environment. The robotic control system is distributed over many interacting behavioral levels with and ABM responsible for learning the sensory-response couplings at each behavioral level. Each behavioral level is established through its sensory and reinforcement signal connections with the environment. When these ABMs and low level actuator systems are assembled using command and reinforcement connections into an adaptive framework, what is referred to as a Distributed Adaptive Control System (DACS) results. This DACS can be considered as the robot's artificial adaptive nervous system. The resulting self-learning and self-repairing DACS is capable of controlling a mobile robot within the bounds established by the robot's physical and sensory configurations. Within the DACS framework learning is accomplished, not with an external teacher suppling direction, but by using goal, environmental and sensory based reinforcements inherently available through the robot's interactions with the environment. The operating characteristics of the robot are then dependent upon the particular environmental conditions in which the robot finds itself and not on the preconceived notions of a human designer. Cooperative and collective behaviors within populations of robots are shown as extensions of the DACS framework. The complex nature of the unaided interactions of a mobile robot within its surrounding environment prevented in this research all but the basic analytical studies. For this reason the proposed DACS method was studied using a simulated quadrupled mobile robot. Useful skills and behaviors were found to emerge at all behavioral levels as they were autonomously discovered and learned. When confronted with unforeseeable changes and malfunctions, the DACS framework was affected to various extents depending upon the severity of the changes. The DACS adapted either the single behavioral level at the origin of the change or other higher behavioral levels as required in response to more sever changes. Although this research was performed in the context of mobile robots, the principles developed in this thesis might be applied to areas such as process control, flexible manufacturing, power transmission and communication systems. (Abstract shortened by UMI.)