Automated plant-scale crop monitoring by a farmer robot using a consumer-grade RGB-D camera

Abstract

This paper investigates the potential of a consumer-grade infrared stereo camera, i.e. the Intel RealSense D435, to automatically extract crop status information, such as Normalized Difference Vegetation Index (NDVI), in arable and permanent crops. The sensing device includes two infrared (IR) sensors for depth calculation and one colour sensor, which provide, for each point of the scene, both IR and visible light information thus making it possible pixel per pixel NDVI estimations. Measurements were performed on various arable crops including corn (Zea mays) and barley (Ordeum vulgare) and on two vine varieties, Freisa and Malvasia, and were compared to measurements taken by a Trimble GreenSeeker handheld crop sensor. Results show that the RealSense camera tends to underestimate NDVI values compared to the GreenSeeker, with squared correlation coefficient r2 = 0.68. The fitted regression equation is successively applied to correct new camera observations, resulting in good agreement with the GreenSeeker output. The use of the RGB-D camera to simultaneously provide canopy height measurements by a farmer robot is also demonstrated in a Malvasia field, showing that the proposed system can be effectively adopted for fully automated plant-scale monitoring of vineyards.