Enhanced deep learning model for apple detection, localization, and counting in complex orchards for robotic arm-based harvesting

摘要

• This study presents an enhanced YOLOv8n model optimized for high-precision robotic apple harvesting in complex orchard environments. • The enhanced YOLOv8n outperforms the original YOLOv8n, YOLOv5, YOLOv6, and Real-Time Detection Transformer across key performance metrics, excelling in dynamic conditions. • Significant improvements in apple localization and counting accuracy make the model more reliable and efficient for robotic harvesting. • These advancements set a new standard in agricultural robotics, offering a highly effective and precise solution for automated harvesting systems. The growing demand for automation in the apple-harvesting industry remains challenging due to the complex and dynamic nature of orchard environments. This study presents an enhanced deep learning model designed to improve the accuracy and adaptability of recognition algorithms for robotic arm-based harvesting. Specifically, an optimized You Only Look Once (YOLO) v8n model was developed by integrating a dilation-wise residual–dilated re-parameterization block module, a generalized feature pyramid network, and the Scylla Intersection-over-Union loss function. The enhanced model was trained and evaluated on a comprehensive dataset, achieving precision, recall, F1 score, and mAP50 values of 81.43%, 68.48%, 74.40%, and 81.68%, respectively. These results indicate improvements of 1.06%, 1.42%, 1.28%, and 1.61% over the original YOLOv8n, while preserving comparable model parameters, computational efficiency, and detection speed. Furthermore, the enhanced model demonstrated superior overall performance compared to YOLOv5, YOLOv6, and RT-DETR. To validate its adaptability and robustness, the enhanced model was rigorously tested against the original YOLOv8n model diverse conditions, including varying growth stage, lighting environments, field of view, and levels of occlusion. In outdoor field experiments conducted under cloudy, low-light, and artificial lighting conditions, the model achieved localization errors of 2.43 mm (X-axis), 3.70 mm (Y-axis), and 1.28 mm (Z-axis), representing reductions of 19.27%, 12.67%, and 23.05%, respectively. Furthermore, counting accuracy improved to 69.39%, reflecting a 2.42% increase over the original model. The results demonstrate the enhanced model's reliable performance and heightened precision for robotic arm-based apple harvesting in complex and challenging orchard environments. The study also provides a comprehensive analysis of the model's strengths, limitations, and avenues for future research. Ultimately, this work contributes to advancing agricultural automation, paving the way for smarter, more efficient, and sustainable farming practices.