Bayesian Optimization for Efficient Tuning of Visual Servo and Computed Torque Controllers in a Reinforcement Learning Scenario

Abstract

Although the search for optimal parameters is a central concern for the design stage of control systems, this adjustment is generally not optimized in the design of visual servo controllers. However, for a classic position-based visual servo controller, the choice of the proportional gain that multiplies the computed error may directly affect the system's performance, and may even lead to instability. On the other hand, adjusting such a parameter can be a time-consuming and hard-working task. Thus, in this work, we propose to automate the search for the linear and angular gains of a visual servo controller through Bayesian optimization. We simulate the environment in Matlab with a Kinova GEN3 7DOF robot in a reinforcement learning scenario, in which the projected cost function is evaluated directly on the robot. We demonstrate that Bayesian optimization is capable of finding the visual servo controller gains, as well as the robot internal controller gains, with up to 13 and 14 times fewer iterations when compared to an on-police actor-critic model-free algorithm and the genetic algorithm respectively. Furthermore, we show that the obtained controller performs better considering different control performance parameters and in qualitative evaluations regarding the Cartesian and image spaces.