High-throughput detection of tomato architectural traits based on UGV plant phenotyping system

Abstract

<b><sc>Abstract.</sc></b> Temporal plant architectural traits in tomato growth are important for understanding genotype by environment interactions and promoting ideotype breeding. While large-scale manual measurement of these traits is time-consuming, labor-intensive and subjective. In this study, we proposed a novel pipeline for high-throughput detection of tomato architectural traits at different growth stages using an unmanned ground vehicle (UGV) plant phenotyping system. The UGV equipped with a robotic arm automatically navigated to the waypoints in turn, and collected RGB-D images and multispectral (MS) images from multiple viewpoints. The SegFormer with fusion of multispectral and depth modalities (MSD-SF) was employed to semantically segment different plant organs from the registered image pairs. Organ point clouds were then generated from these masked images and clustered into instances. Finally, six key architectural traits, including fruit spacing (FS), inflorescence height (IH), stem thickness (ST), leaf spacing (LS), total leaf area (TLA), and leaf inclination angle distribution (LIAD) were extracted at different growth stages of tomato plants. Field tests show that the UGV was able to complete the operation of a 320-square-meter greenhouse in 2.5 h, with eight viewpoints per plant covered in 30 s. The mean intersection over union (mIoU) of MSD-SF on the test dataset was 75.12%. Compared to the available ground truth, the root mean square errors (RMSEs) of the estimated FS, IH, ST and LS were 0.014 m, 0.043 m, 0.003 m and 0.015 m, respectively. The visualization results of the estimated TLA and LIAD were consistent with the actual growth trends of tomato plants. The results indicate that the proposed approach is promising for high-throughput phenotyping of tomato architectural traits in the greenhouse, which would advance the precision and efficiency for both crop breeding and production management.