An Improved Q-learning Algorithm for Path-Planning of a Mobile Robot

Abstract

Classical Q-learning requires huge computations to attain convergence and a large storage to save the Q-values for all possible actions in a given state. This paper proposes an alternative approach to Q-learning to reduce the convergence time without using the optimal path from a random starting state of a final goal state, when the Q-table is used for path planning of a mobile robot. Further, the proposed algorithm stores the Qvalue for the best possible action at a state, and thus save significant storage. Experiments reveal that the acquired Q-table obtained by the proposed algorithm helps in saving turning angles of the robot in the planning stage. Reduction in turning angles is economic from the point of view of energy consumption by the robot. Thus the proposed algorithm has several merits with respect to classical Q-learning. The proposed algorithm is constructed based on four fundamental properties derived here and the validation of the algorithm is studied with Khepera-II robot.