Hyperspectral Vision-Based Machine Learning for Robust Plant Recognition in Autonomous Weed Control.

Abstract

While herbicide application and mechanical cultivation remain the primary means for weed control in agricultural production, the only solution to-date for weed control within close proximity of crop plants in the seedline is hand hoeing. To reduce manual labor cost and minimize herbicide usage for organic farming, this research developed an intelligent robotic system for automated weed detection and control within the seedline. The system utilized visible and near infrared (NIR) reflectance-based features in hyperspectral images of plant foliage for real-time species recognition. This technique is less computationally intensive than the shape- and texture-based machine vision methods. For real-time, in-field applications, this technique is superior to traditional shape-based machine vision, as it does not require singulation of individual plants, and is robust to visual occlusion and less susceptible to leaf morphological variation or damage. A principle challenge of reflectance-based species recognition has been that the optical properties of plant foliage are functions of external growing conditions, and are greatly impacted by the variability in natural environment of agricultural fields. Prior studies of using visible and NIR spectroscopy for plant recognition were restricted to reflectance spectra measured on plants grown in a single season with exposure to a single environmental condition. This work, for the first time, demonstrated the potential of hyperspectral imaging technology for plant species identification under varying external factors of growing temperature, soil moisture and sunlight intensity, as well as over three multiple seasons of natural field environment. This work also developed adaptive learning techniques that mitigated environmental effects and provided solutions to robust plant recognition across the variation in the studied single conditions and seasons. Finally, the machine vision system was coupled with a thermal micro-dosing application system and validated under outdoor conditions for real-time automated weed control with heated food-grade oil. This research provided a complete solution to automated weed control in row crops using machine vision reflectance-based plant recognition. The technical results of this research are summarized as below: Multispectral Bayesian classifiers were developed for distinguishing tomatoes among black nightshade and pigweed. The effects of variation in the three single environmental factors of temperature, soil moisture, and solar irradiance, on spectroscopy-based plant recognition demonstrated that (i) the optimum performance, ranging from 88.2% to 95.3%, occurred when the models were applied in same environmental conditions as represented in training; (ii) increasing the deviation of the validation conditions from the calibration conditions degraded the performance to 62.5–81.7%; (iii) environmental stress made the plant species more distinguishable and slightly improved the overall accuracy by 1.3–6.9% for same condition applications; (iv) the Bayesian classifiers optimized for the normal conditions demonstrated more robust plant recognition over environmental variations. An environmentally-adaptive machine learning algorithm was developed for automatic site-specific recalibration of a Bayesian classifier in a dynamic environment. Validation performance of the classifier demonstrated that site-specific recalibration can be implemented by establishing the models exclusively with a fraction of new data (approximately 30 to 80 plants) without adverse impact due to ignoring the old data originally used to train the model. This method alleviated the bias produced by single-condition calibration and, overall, it improved the classification rates to 90.4–94.5% across variation in the three studied environmental factors. Global calibration was another approach investigated to improve robustness of the Bayesian classifiers to varying environmental conditions o