Machine vision for a selective broccoli harvesting robot

Abstract

The selective hand-harvest of fresh market broccoli is labor-intensive and comprises about 35% of the total production costs. This research was conducted to determine whether machine vision can be used to detect broccoli heads, as a first step in the development of a fully autonomous selective harvester. A texture and color based image segmentation was used to separate the broccoli head from the background. Segmentation results were compared to a ground truth dataset of 200 images. In these images, 228 broccoli heads of varying sizes were classified by two human experts with the GrabCut algorithm. Image segmentation was evaluated by two different metrics. The first was a pixel-based spatial overlap between the ground truth classification and image segmentation, which resulted an average overlap of 93.8%. The second metric was the individual broccoli head detection and the corresponding confusion matrix. These showed a precision score of 99.5%, indicating only one false positive. The specificity was 97.9%, negative predictive value was 69.7% and overall accuracy 92.4%. In total, 208 broccoli heads were detected by the machine vision software, indicating a sensitivity score of 91.2%. The average pixel size of the non-detected heads was smaller than the pixel size of the detected heads.