A Deep Learning-Based Stalk Grasping Pipeline

Abstract

The need for fast and precise measurements of plant attributes makes robotic solutions an ideal replacement for labor-intensive phenotyping processes. In this work we present a deep learning-based high throughput, online pipeline for in-situ sorghum stalk detection and grasping. We use a variation of Generative Adversarial Network (GAN) for stalk segmentation trained on a relatively small number of images followed by a grasp point generation pipeline. The presented pipeline is robust to field challenges such as occlusions, high stalk density and lighting variation, and was deployed on a custom-built ground robot. We tested our end-to-end system in a field of Sorghum bicolor in South Carolina, USA, achieving an average grasping accuracy of 74.13% and a stalk detection F1 score of 0.90. Grasp point detection for plant manipulation takes an average of 0.98 seconds, and pixel-wise stalk detection takes 0.2 seconds per image.