A Smart Multimodal System for Crop Disease Detection and Agricultural Decision Support

Abstract

Precision agriculture has become an important research area for improving crop productivity, food security, and sustainable farming practices. Crop diseases significantly reduce agricultural yield and economic stability, especially in developing agricultural economies. Traditional disease detection methods rely heavily on manual inspection and expert knowledge, which are time-consuming and often inaccurate under large-scale farming conditions. This manuscript proposes a multimodal artificial intelligence framework that integrates Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), environmental parameters, and reasoning-based support systems for crop disease identification and agricultural decision support.

The proposed system combines crop leaf images, weather conditions, and environmental sensor data to improve disease classification performance. The PlantVillage dataset containing approximately 54,000 crop leaf images across multiple disease categories was utilized for experimentation. Image preprocessing techniques such as resizing, normalization, augmentation, and noise removal were applied before model training. A hybrid CNN-Transformer architecture was implemented using Adam optimizer with a learning rate of 0.001 over 50 epochs. Experimental evaluation demonstrated an overall classification accuracy of 93.2%, outperforming traditional CNN-only approaches.
The system also provides treatment recommendations and environmental analysis to support farmers in real-time decision-making. Comparative evaluation using precision, recall, F1-score, and confusion matrix analysis confirms the effectiveness of multimodal data fusion. Although computational complexity and deployment challenges remain limitations, the proposed framework demonstrates strong potential for smart agriculture applications.