Behavioral Optimization in a Robotic Serial Reaching Task Using Predictive Information

Abstract

Prediction is a powerful approach to minimize errors and control problems in familiar environments and tasks. In human motor execution of sequential action, context effects can be observed, such as anticipation of or predictive movement toward target objects, where later subactions are affected by the execution of earlier subactions. In this article, we present a simulation framework for a serial reaching task using a 4-DoF robotic arm to examine the learning of context effects in simulated robotic reinforcement learning agents. As we demonstrate, giving robotic agents access to predictive information about a future target object’s identity results in motion optimization, where the identity of the next target modulates earlier subactions. Specifically, agents learn to anticipate and predict the location of the next target object, and move toward it before it appears, thus achieving higher rewards than agents that were not given predictive information.