Fusion between a color camera and a TOF camera to improve traversability of agricultural vehicles

摘要

Precision agriculture vehicles need autonomous navigation in cultures to carry out their tasks. The detection of obstacles in crop row is crucial for safe navigation. Lidar devices or Time Of Flight cameras (TOF), allow to obtain\ngeometric data in natural environment. Fusion of those geometric information with a color camera data, improves the identification and discrimination of objects in agricultural cultures. Autonomous agricultural vehicles can use these\ngeometric and colorimetric data in real time, to follow crop rows and detect obstacles while executing various precision agriculture operations, such as monitoring tasks, maintenance or harvesting, for preserving safety, for both vehicle and environment. In our application, a lidar and a TOF camera which detected objects respectively on a plane and in 3D space, were associated with a color camera. Sensors were embedded on a mobile robot. Fusion method consisted to put the lidar and TOF data inside the color image, and make colorimetric and geometric measures on the detected objects (grass, leaf, tree branches), in front of vehicle, for identifying these ones. Traversability operations, tested with a mobile robot in vineyard environment, during crop raw tracking operations, consisted to take a decision, according to the size and nature of the detected objects, working under different lighting conditions, and various vegetation levels: the vehicle crossed the object at reduced speed, or it avoided it, or it stopped. Experimentations showed the effectiveness of the approach to detect and identify objects, such as grass or solid obstacles, at various heights from the ground.