Ai-Based Real-Time Filipino Sign Language Recognition and Translation System with Context-Aware Natural Language Processing

Abstract

The Filipino Sign Language (FSL) serves as the official sign language of the Deaf community in the Philippines, as recognized under Republic Act 11106. Despite this recognition, communication barriers persist in educational institutions, healthcare facilities, workplaces, and other settings where Deaf and hearing individuals interact, largely due to the limited understanding of FSL among the hearing majority. Existing sign language recognition applications predominantly target other sign languages such as American Sign Language and are constrained to isolated gesture recognition, failing to address the continuous and dynamic nature of Filipino Sign Language communication. This study developed an AI-Based Real-Time Filipino Sign Language Recognition and Translation System with Context-Aware Natural Language Processing, implemented as an Android mobile application. The system employs MediaPipe Holistic for coordinate-based 3D landmark extraction from hand and body pose, and a Long Short-Term Memory (LSTM) neural network for sequential gesture pattern recognition. A dataset of 20 FSL vocabulary items was collected from seven Filipino Sign Language signers, and a four-technique data augmentation pipeline including Gaussian noise, time shifting, scale variation, and hand mirroring was applied to improve model generalization across different signing styles and dominant hand preferences. The trained model was converted to TensorFlow Lite format for efficient on-device inference without requiring server-side computation or internet connectivity. The system achieved an overall classification accuracy of 97% on the evaluated vocabulary, demonstrating the viability of real-time FSL gesture recognition on consumer-grade mobile hardware. The study adopted a developmental research design with an Agile software development approach to facilitate iterative refinement throughout the development process. The results confirm that the proposed MediaPipe-LSTM architecture provides a practical and accessible foundation for bridging the communication gap between the Filipino Deaf community and the hearing population.