Olfactory-Based Navigation via Model-Based Reinforcement Learning and Fuzzy Inference Methods

Abstract

This article presents an olfactory-based navigation algorithm for using a mobile robot to locate an odor source in a turbulent flow environment. We analogize the odor source localization as a reinforcement learning problem. During the odor plume tracing process, the belief state in a partially observable Markov decision process model is adapted to generate a source probability map that estimates possible odor source locations, and a hidden Markov model is employed to produce a plume distribution map that premises plume propagation areas. Both source and plume estimations are fed to the robot, and a decision-making approach based on fuzzy inference is designed to dynamically fuse information from two maps and to balance the exploitation and exploration of the search. After assigning the fused information to reward functions, a value iteration based path planning algorithm is presented to solve for the optimal action policy. Comparing to other commonly used olfactory-based navigation algorithms, such as moth-inspired and Bayesian inference methods, simulation results show that the proposed method is more intelligent and efficient.