EDSC-HRAFNet: An apple tree branch semantic segmentation model for harvesting robots under complex orchard conditions

Abstract

Accurate semantic understanding of tree branches is critical for orchard harvesting robots in automated fruit harvesting and pruning. Existing methods suffer from low detection accuracy and limited adaptability in complex orchard environments. This paper proposes a novel apple tree branch semantic segmentation model (EDSC-HRAFNet) for orchard harvesting robots under complex orchards conditions. The presented Enhanced Dynamic Snake Convolution module (EDSC_unit) is integrated into High-Resolution Network (HRNet) backbone to extract topological features such as branching points and bifurcations. Then, the HeteroFPN module is designed as the Neck structure, and performs semantic-position information cyclic interaction on the multi-level output features of Backbone in a dual-path collaborative framework, obtaining multi-level features with stronger comprehensive representation capabilities. And the Parallel-M4 Decode module is designed for the network head, performing parallel processing based on the characteristics of features at different levels. This framework could concatenate the features to generate geometrically precise segmentation masks. Finally, we constructed a dataset of in-situ apple trees under diverse real-world conditions to verify the performance and superiority of EDSC-HRAFNet. EDSC-HRAFNet demonstrates state-of-the-art segmentation performance across eight challenging orchard scenarios and exhibits robust generalization. Experiments show that proposed model achieves precision, recall, dice, IoU, MIoU, and MPA of 91.50%, 91.71%, 91.60%, 84.51%, 91.72%, and 95.58%, respectively. These improvements are 6.98%, 10.09%, 9.21%, 13.5%, 7.25%, and 5.24% compared to HRNet. Compared with existing models including Pspnet, Deeplabv3+ series and Unet series, the precision, recall, IoU is improved by 13.92% to 29.61%, 23.59% to 42.38% and 28.79% to 46.67% respectively. EDSC-HRAFNet's superior branch segmentation capability in challenging orchard environments provides a practical foundation for robotic automation in agricultural orchards. • This paper proposes a novel apple tree branch semantic segmentation model (EDSC-HRAFNet) for orchard harvesting robots under complex orchards conditions. • The presented EDSC_unit module is integrated into HRNet backbone repeat multi-scale fusion to better capture slender branch structures and complex tree morphology. • Proposed HeteroFPN provides topologically-aware features for branch segmentation. Cyclic semantic-localization interaction significantly improves bifurcation and thin-end perception. • The designed Parallel-M4 Decoder preserves extreme-scale feature integrity, optimizes mid-high-level geometric consistency via parallel processing, and enhances topological adaptability. • Experiments show that EDSC-HRAFNet achieves Precision, Recall, Dice, IoU, MIoU, and MPA of 91.50%, 91.71%, 91.60%, 84.51%, 91.72%, and 95.58%, respectively.