Development of an Enhanced Financial Fraud Detection Model using a Machine Learning Algorithm

Abstract

This study presents an enhanced financial fraud detection system that uses unsupervised machine learning techniques to identify anomalous financial transactions in highly imbalanced datasets. The increasing sophistication of digital financial fraud has reduced the effectiveness of traditional rule-based security systems, thereby necessitating adaptive and intelligent detection approaches. The proposed system uses the Isolation Forest algorithm for its computational efficiency and ability to detect previously unseen fraud patterns without relying on labelled data. A large transaction dataset comprising 284,807 financial records was preprocessed with normalization and dimensionality-reduction techniques prior to model training and evaluation. The developed model was integrated into a Flask-based web application that supports real-time transaction analysis via a user-friendly dashboard. Experimental results demonstrated strong predictive performance, achieving a recall of 94.7%, a precision of 53.5%, an F1 Score of 68.6%, and an ROC-AUC of 0.91. Comparative evaluation with One-Class SVM and Autoencoder models further revealed that while Autoencoders achieved slightly higher overall discrimination performance, Isolation Forest provided a more computationally lightweight and deployment-efficient solution for resource-constrained financial environments. The findings also highlight the practical implications of false positive management in fraud detection systems and emphasize the importance of balancing fraud sensitivity with operational efficiency. Overall, the study confirms that lightweight unsupervised anomaly detection models provide an adaptive, scalable, and deployable solution for enhancing financial security in modern fintech ecosystems.