Performance evaluation in obstacle avoidance

Abstract

No quantitative procedure currently exists to evaluate the obstacle avoidance capabilities of robotic systems. Such an evaluation method is not only needed to compare different avoidance methods, but also to determine the operational limits of autonomous systems. This work proposes an evaluation framework which can find such limits. The framework comprises two types of tests: detection tests and avoidance tests. For each type, both environment and performance metrics need to be defined. For detection tests such metrics are well known, but for avoidance tests such metrics are not readily available. Therefore a new set of metrics is proposed. The framework is applied to a UAV that uses stereo vision to detect obstacles. Three different avoidance methods are compared in environments of varying difficulty.