A lightweight keypoint detection model-based method for strawberry recognition and picking point localization in multi-occlusion scenes

摘要

Strawberries grown on elevated stands usually suffer from fruit occlusion issues, which severely limit the implementation of strawberry recognition and picking point localization, and the embedded devices carried by strawberry picking robots have high requirements for model lightweighting, posing a dual challenge to the efficient execution of automated picking tasks by robots. To address this issue, this study proposes a method for strawberry recognition and picking point localization in multi-occlusion scenes based on a lightweight keypoint detection model. Firstly, a strawberry dataset covering no, slight, moderate, and heavy occlusion scenes is constructed. Then, a lightweight strawberry recognition and keypoint detection network, LS-net, is proposed. LS-net improves the spatial relationship modelling capability between strawberries and stems by integrating the lightweight MobileNetv4 backbone with the Mobile Grouped-Query Attention mechanism; improves the feature pyramid network using depthwise separable convolutions and incorporates an anchor-free decoupled head network to reduce computational complexity while maintaining detection accuracy; and introduces the Matrix Non-Maximum Suppression to optimize the processing of overlapping strawberries, which effectively reduces the false negative detections. Based on the keypoint detection results from LS-net, the picking point coordinates and stem pose are calculated after a series of processes such as region-of-interest extraction, binarization, and depth data alignment. The experimental results show that the accuracy of LS-net is 91.07 %, the mean average precision is 93.93 %, and the average pixel Euclidean distance is 4.79. By deploying LS-net to the embedded device, its frames per second reaches 78.2, and the success rates of 3D picking point localization and stem pose estimation are 84.07 % and 81.32 %, respectively. LS-net and related methods provide a visual recognition solution adapted to embedded devices for strawberry picking robots. • Proposed a lightweight model (LS-net) for strawberry recognition with occlusion. • 3D strawberry picking point located and stem pose estimated via depth data. • LS-net accurately recognizes strawberries under different occlusion levels. • Lightweight model enables efficient inference on embedded devices.