Multi-spectral image transformer descriptor classification combined with molecular tools for early detection of tomato grey mould

摘要

Accurate early detection of plant diseases is a critical milestone in the development of self-navigating robots for precision agriculture and still remains one of the most significant challenges. Plant pathogens, like Botrytis cinerea causing grey mould disease, pose significant threats to agriculture and food safety. This study focuses on the early detection of B. cinerea in tomato crops using Artificial Intelligence. Specifically, the Deep Learning (DL) YOLOv8 architecture was employed to apply single class segmentation on plant captures to extract the tomato leaves locational information. Each plant capture includes five images from hyperspectral wavelengths (at 460, 540, 640, 775 and 875 nm) and one RGB images. The leaf segmentation achieved 81.7 % Mean Average Precision (mAP) at Intersection over Union (IoU) threshold 0.5 (mAP50). Then the for each leaf segment, and array of descriptors is extracted through various Transformer models. Finally, the descriptors are classified through a KNN or an LSTM solution and the results of all the descriptors for each leaf from each of the Transformer models, for each of the six images of the capture, are ensembled to yield the class of each leaf. The best ensemble results were produced by only accumulating results from some of the Transformer and specifically the MaxViT-L, the ViT-B (P:16 × 16 – C), the VOLO (D:5) and the XCIT-L (L:24 – P:16 × 16), by only using the results provided by the images at the hyperspectral wavelengths at 540, 640 nm and the RGB image, and by applying the LSTM solution. The process achieved a 79.41 % classification accuracy and 71.12 % F1-Score for all classes. Additionally, early, rapid and accurate detection of grey mould at the early stages of the infection or at latent infections when symptoms are not visible was assessed by comparative qPCR-based methods (qRT-PCR) for fungal biomass estimation in plant tissues and the rest of this study's methodology. The findings of this study represent a significant advancement in crop disease detection and management, and highlight the potential for integrating these methods into on-site digital systems (robots, mobile apps, etc.) under real-world settings. The results, demonstrate the effectiveness of combining segmentation and transformer-based classification models for early and accurate detection of grey mould disease.