Adaptive Experience Sampling for Motion Planning Using the Generator-Critic Framework

摘要

Sampling-based motion planners are widely used for motion planning with high- <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dof</small> robots. These planners generally rely on a uniform distribution to explore the search space. Recent work has explored learning biased sampling distributions to improve the time efficiency of these planners. However, learning such distributions is challenging, since there is no direct connection between the choice of distributions and the performance of the downstream planner. To alleviate this challenge, this letter proposes <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">apes</small> , a framework that learns sampling distributions optimized <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">directly</i> for the planner's performance. This is done using a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">critic</i> , which serves as a differentiable surrogate objective modeling the planner's performance – thus allowing gradients to circumvent the non-differentiable planner. Leveraging the differentiability of the critic, we train a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">generator</i> , which outputs sampling distributions optimized for the given problem instance. We evaluate <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">apes</small> on a series of realistic and challenging high- <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dof</small> manipulation problems in simulation. Our experimental results demonstrate that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">apes</small> can learn high-quality distributions that improve planning performance more than other biased sampling baselines.