Fruity: A Multi-modal Dataset for Fruit Recognition and 6D-Pose Estimation in Precision Agriculture

Abstract

The application of robotic platforms for precision agriculture is gaining traction in modern research. However, the demand for a complete fruit dataset is still not satisfied. In this paper, we present fruity, a multi-modal fruit dataset with a variety of use cases such as 6D-pose estimation, fruit detection, fruit picking applications, etc. To the best of our knowledge, this dataset is the first-ever multi-modal fruit dataset tailored specifically for fruit 6D pose estimation in precision agriculture. The dataset is collected over a range of multiple sensors consisting of an RGB-D camera, thermal camera and an indoor tracking camera for ground truth poses. Fruity features RGB images, stereo depth images, thermal images, camera 6Dposes, fruit 6D-poses and relative 6D-poses between the cameras and fruits. The classes of the dataset are commonly harvested fruits which include: apples, oranges, bananas, avocados and lemons. It is also enriched with a clustered class to account for occlusion scenario. The dataset is recorded over multiple trajectories implemented with multiple platforms encompassing a robotic manipulator and an Unmanned Aerial Vehicle (UAV). The dataset alongside the documentation and utility tools is publicly available at: https://github.com/MahmoudYidi/Fruity.git.