AI-based framework for early detection and segmentation of green citrus fruits in orchards

Abstract

The detection and segmentation of tiny green citrus fruits in dense orchards play a vital role in modern farming, directly influencing yield prediction, resource management, and timely decision-making. This research presents a cutting-edge framework that combines Multiscale Vision Transformers version 2 (MViTv2) with Cascade Mask R-CNN to tackle these challenges effectively. By extending the focus from close-up images to the novel inclusion of full-tree images, the framework enables accurate early-stage detection, segmentation, and counting of citrus fruits in practical orchard settings. Unlike conventional methods, this approach uses a dual-image strategy: close-up images for training and full-tree images—more complex due to dense foliage and small fruits—for testing and real-world applications. To enhance detection accuracy in these detailed, full-tree images, the framework employs an innovative image-slicing method, breaking high-resolution images into smaller parts to capture finer details. The model was tested on a unique dataset featuring citrus orchards of three varieties: Nules grafted on Volka, Sidi Aissa grafted on Volka, and Orogrande grafted on sour orange. Results showed that the MViTv2_L backbone outperformed alternatives, achieving a mean Average Precision (mAP) of 72.97% for bounding boxes and 84.40% for masks. The image-slicing technique further boosted fruit detection in full-tree images, achieving an R 2 value of up to 0.81 for fruit counting. This dual-image method, paired with advanced segmentation and detection technologies, marks a significant step forward for agricultural robotics and precision farming, enabling accurate early-stage fruit detection in real-world orchard environments. • Introduced a novel framework combining Cascade Mask R-CNN with the MViTv2.L backbone, significantly enhancing citrus fruit detection and segmentation accuracy. • Implementing an innovative slicing strategy improved the model's ability to manage dense foliage and overlapping fruits, leading to more precise fruit counts. • A custom dataset of high-resolution images of citrus orchards was created using two different image capture protocols to improve segmentation and counting reliability in dense foliage. • Our enhanced detection and segmentation capabilities offer practical applications in agricultural technology, providing a robust tool for accurate crop analysis and management.