Robotic Plot-scale Peanut Counting and Yield Estimation using LoFTR-based Image Stitching and Improved RT-DETR

摘要

<b><sc>Abstract.</sc></b> Peanuts, ranking as the seventh-largest crop in the United States with a farm value exceeding $1 billion, are pivotal to global food security. Conventional peanut yield estimation methods involve digging, harvesting, transporting, and weighing, which are labor-intensive and inefficient for large-scale operations. This inefficiency is particularly pronounced in peanut breeding, where requires precise yield estimations of each plot-scale pods for genotypes comparison and selection. We proposed an automated approach utilizing a robotic system equipped with machine vision to predict peanut yields post-digging and inverting. This system leverages a mobile robot with an imaging system that captures sequential images of peanut plots, each representing a different genotype, utilizing spatial geographic information. A robust hierarchical strategy was introduced for plot-scale image stitching, employing a Local Feature Transformer (LoFTR)-based feature matching algorithm. Additionally, the Real-Time Detection Transformer (RT-DETR) was customized for pod detection by integrating partial convolution into a lightweight ResNet-18 backbone and refining the upsampling and downsampling modules in Cross-scale Feature Fusion. Our methods were validated in two breeding fields, where the LoFTR-based stitching achieved approximately three times denser and more uniform feature matching than the conventional Scale-Invariant Feature Transform (SIFT) approach. The customized peanut pod detector demonstrated a mean Average Precision (mAP50) of 89.3% and an mAP95 of 55.0% with lighter weights and less computation, improving by 3.3% and 5.9%, respectively, over the original RT-DETR model. Finally, we deployed the detector on the stitched plot-scale images and calculated the pods number for predicting the yield. Achieving a Mean Absolute Percentage Error (MAPE) of 9% and an R-square of 0.47, our approach outperforms the mainstream Structure from Motion (SfM) based methods. This innovative approach significantly reduces the time and labor required for yield determination, thereby advancing the efficiency of peanut breeding operations in complex, dynamic outdoor environments.