Facial Expression and Gesture Recognition System for Stress Detection with Deep Learning

Abstract

Stress is a significant contributor to declining mental and physical health, necessitating reliable and non-intrusive methods for early detection and continuous monitoring. This study proposes a deep learning–based framework for automated stress detection using facial expression and gesture recognition. Unlike traditional stress assessment methods that rely on self-reported surveys or physiological sensors, the proposed approach leverages visual behavioral cues to enable real-time, contactless monitoring.
The system integrates a Convolutional Neural Network (CNN) for spatial feature extraction from facial images and a Long Short-Term Memory (LSTM) network for modeling temporal dependencies in gesture sequences. Benchmark facial expression and gesture datasets were utilized for training and validation. Data preprocessing included normalization, augmentation, and structured dataset splitting to enhance model generalization. Performance evaluation was conducted using accuracy, precision, recall, F1-score, and root mean squared error (RMSE).
Experimental results indicate that the proposed CNN–LSTM architecture effectively captures subtle stress-related patterns in visual data, demonstrating strong classification performance. The findings support the feasibility of visual-based stress detection as a scalable and non-invasive alternative to physiological monitoring systems. While limitations remain regarding dataset diversity and real-world variability, the study establishes a foundation for future multimodal and real-time stress detection systems applicable in healthcare, workplace monitoring, and human–computer interaction contexts.