The week's pick
Random Articles
Reseach Article
Real-Time Sign Language Detection using MediaPipe and Deep Learning
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 187 - Number 57 |
| Year of Publication: 2025 |
| Authors: Tanishka Chakraborty, Souryadip Ghosh, Anita Pal, Dhrubajyoti Ghosh |
10.5120/ijca2025925952
|
Tanishka Chakraborty, Souryadip Ghosh, Anita Pal, Dhrubajyoti Ghosh . Real-Time Sign Language Detection using MediaPipe and Deep Learning. International Journal of Computer Applications. 187, 57 ( Nov 2025), 78-83. DOI=10.5120/ijca2025925952
Abstract
Sign language serves as a crucial communication medium for individuals with hearing and speech impairments. However, effective real-time sign language recognition remains a challenging task due to variations in hand gestures, environmental factors, and computational constraints. This paper proposes a robust and efficient real-time sign language detection system that leverages advanced computer vision and deep learning techniques to recognize hand gestures accurately. The system integrates Media Pipe, a state-of-the- art hand tracking framework, to extract precise hand landmarks, which are then processed and classified using a deep learning model trained with TensorFlow. The model is developed using a dataset collected through OpenCV, ensuring a comprehensive representation of various sign language gestures. To enhance user interaction and accessibility, the system incorporates text-to-speech (TTS) technology, enabling the real-time conversion of recognized gestures into spoken words. This feature significantly improves communication for individuals who rely on sign language, bridging the gap between non-verbal and verbal communication. Extensive experimentation and evaluation demonstrate the system's high accuracy and efficiency in real- time gesture recognition. By employing an optimized approach that balances computational performance and recognition accuracy, the proposed system offers a cost- effective, scalable, and reliable solution for assisting individuals with speech and hearing disabilities. Furthermore, the lightweight and real-time nature of this approach makes it suitable for deployment on various platforms, including personal computers, mobile devices, and embedded systems. The findings of this study highlight the potential of integrating artificial intelligence and computer vision for assistive communication technologies. Future work aims to expand the system's capabilities by incorporating a larger vocabulary of gestures, enhancing generalization across diverse user demographics, and optimizing the model for improved real- world deployment.
References
- Kumar, R., Bajpai, A., and Sinha, A., 2023, Mediapipe and CNNs for Real-Time ASL Gesture Recognition, arXivLabs, DOI: 10.48550/arXiv.2305.05296
- Verma, R., A., Singh, G., Meghwal, K., Ramji, B., and Dadheech, K., P., 2024, Enhancing Sign Language Detection through Mediapipe and Convolutional Neural Networks (CNN), arXivLabs,, DOI: 10.48550/arXiv.2406.03729
- Jaju, R., and Karwa, P., 2023, Sign Language Detection of English Alphabets for Deaf and Dumb People, International Journal for Research in Applied Science & Engineering Technology (IJRASET), 11(Dec. 2023), 314-318
- Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., Grundmann, M., 2019, BlazeFace: Sub- millisecond Neural Face Detection on Mobile GPUs, DOI: 10.48550/arXiv.1907.05047
- Honesty Praiselin,H., J., E., Manikandan, G.,Veronica,V., Hemalatha, S., 2024, Sign Language Detection and Recognition Using Media Pipe and Deep Learning Algorithm, International Journal of Scientific Research in Science and Technology, (IJSRST), 11 (Apr., 2024), 123-130, DOI: 10.32628/IJSRST52411223
- Zhou, X., Wang, Q., Zhang, C., and Liu. W., 2020, Object Detection with Bounding Box Regression, IEEE Access, 8(2020), 29786–29795, DOI: 10.1109/ACCESS.2020.2978965
- Kim, H., Kum,D., and Chung, J.,M., 2019, Gesture Recognition for Human–Robot Interaction using Mediapipe and CNN, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), DOI: 10.1109/ICCV.2019.00345
- Wang,J., Chen, C., and Liu, X., 2021, Real-Time Hand Gesture Recognition Based on Deep Neural Networks, IEEE Transactions on Neural Networks and Learning Systems, 32(2021), 1980–1992, DOI: 10.1109/TNNLS.2021.3093482
- Kim, H., Kum, D., and Chung, J., M., 2019, Sign Language Recognition Using Modified Deep Learning Network, International Journal of Advanced Computer Science and Applications (IJACSA), 10(2019), 123–130.
- Verma,R., A., Singh, G., Meghwal,K., Ramji, B., and Dadheech, K., P., 2024, Enhancing Sign Language Detection through Mediapipe and Convolutional Neural Networks (CNN), International Journal of Computer Applications, 187(2024), DOI: 10.48550/arXiv.2406.03729
Index Terms
Keywords