Blockchain-Based Multimodal Misinformation Detection System

Abstract

Digital misinformation rapidly spreading exponentially through social media platforms has generated the dire need to have a strong, correct, and transparent detection system in place. The current methods are mostly based on single-model machine learning methods on text-based information, which is not effective in detecting multimodal falsehoods like deceptive memes that integrate images with text. Moreover, centralized storage systems make such systems vulnerable to manipulation of data and they are not very transparent. The given paper features a proposal of a Blockchain-Based Multimodal Misinformation Detection System, which incorporates the use of Convolutional Neural Network (CNN)-based image feature extraction, Optical Character Recognition (OCR), transformer-based text encoding, and feature fusion to classify memes. A purpose-built Fact-Checking Module uses Large Language Model (LLM) reasoning in conjunction with real-time news retrieval to contextually verify textual assertions. An explainability layer calculates the sentiment, toxicity, readability, and perplexity scores to increase the interpretability. Final prediction results are then transformed into SHA-256 cryptographic hashes and stored forever in an Ethereum blockchain as smart contracts, which guarantee auditability immutability. It follows the MERN stack upon Flask-based AI microservice to implement the system. The validation accuracy, the F1-score, and the precision of the traditional unimodal baselines are lower than those of the experimental evaluation (93.9% and 0.935, respectively). The suggested framework will bring the state-of-the-art of multimodal detection, contextual fact verification, explainable AI, and decentralized verification into a single and scalable solution.