D3-TD3: Deep Dense Dueling Architectures in TD3 Algorithm for Robot Path Planning Based on 3D Point Cloud

摘要

Twin delayed deep deterministic (TD3) policy gradient has several limitations when applied in planning a path in environment with a number of dilemmas according to our experiment, due to the complexity of the robot path planning task, the rate of convergence of TD3 algorithm is slow and the rate of collision is high. To address this problem, deep dense dueling twin delayed deep deterministic (D3-TD3) architecture is proposed, a method that preserves important information from cross-layer inputs through dense connections and divides the network into a value function and a dominance function, thus, allowing for faster convergence when solving complex tasks. Finally, a spatial model based on three-dimension (3D) point cloud is built, and simulation experimental results show that in static environment, the algorithm proposed in the paper has 40.6% fewer collisions compared to TD3, 30% fewer collisions compared to TD3-BC, 19.2% fewer collisions compared to Dueling TD3 and 17.4% fewer collisions compared to deep dense TD3. In dynamic and static environment, the algorithm proposed in the paper has 34.4% fewer collisions compared to TD3, 24% fewer collisions compared to TD3-BC, 6% fewer collisions compared to Dueling TD3 and 25% fewer collisions compared to deep dense TD3.