Autonomous Fruit Harvester with Machine Vision

Abstract

This study presents an autonomous fruit harvester with a machine vision capable of detecting and picking or cutting an orange fruit from a tree. The system of is composed of a six-degrees of freedom (6-DOF) robotic arm mounted on a four-wheeled electric kart. The kart uses ZED stereo camera for depth estimation of a target. It can also be used to detect trees using the green detection algorithm. Image processing is done using Microsoft Visual Studio and OpenCV library. The x & y coordinates and distance of the tree are passed on to Arduino microcontroller as inputs to motor control of the wheels. When the kart is less than 65cm to the tree, the kart stops and the robotic arm system takes over to search and harvest orange fruits. The robotic arm has a webcam and ultrasonic sensor attached at its end-effector. The webcam is used for orange fruit detection while ultrasonic sensor is used to provide feedback on the distance of the orange fruit to end-effector. Multiple fruit harvesting is successfully done. The success rate of harvesting and putting fruit into the basket is 80% and 85% for the gripper end-effector and cutter end-effector respectively.