Fruits Detection and Distance Estimation using RGB-D camera for Harvesting Robot

Abstract

In this paper, fruit detection and distance estimation from detected fruits are processed in parallel based on a low-cost and compact RGB-D camera. Fruits are detected in RGB images and surrounded by bounding boxes using modified deep-learning models, which are optimized for high accuracy based on limited computational resources. Distances from the detected fruits to the camera as well as the size of those fruits are estimated based on their respective boxes in depth images, which provided third-dimensional knowledge or 3D localization of the fruits. The proposed method is implemented on an embedded computer connected to an RGB-D camera and validated in the real environment. Experimental results show that high accuracy in detection, distance, and fruit’s size estimation has been achieved. The promising results enable the further grasping action for a harvesting robot.