Development of a Sign Language E-Tutor using Convolutional Neural Network
Abstract
Deaf and hearing-impaired people typically use sign language as their primary form of communication. This study designed a Convolutional Neural Network-based Sign Language e-tutor which removes language barriers between people who are deaf and use sign language and people who can hear and speak. Thus giving deaf people a way to communicate with hearing people in real time, with no need to write notes or use a human sign language interpreter. The method used is comprised of four major phases: data collection, data preprocessing, model training and model evaluation. The Model Precision, Accuracy, Recall and f1 score were 0.977, 0.985, 0.99, and 0.99 respectively.References
Akin-ponnle, A. E. (2021). Cloud - Based Human Sign Language Digit Classification Using CNN : A Case Study of. 7(5), 3899–3903.
Amsaad, F., Prasanna, P.,Pravallika, T., Mamatha, G., Raviteja, B., Lakshmi, M., Alsaadi, N., and Tashtoush, Y. (2023). Toward Secure and Efficient CNN Recognition with Different Activation and Optimization Functions. 10.1007/978-3-031-33743-7_45.
Bayati, M., and Hussein, K. (2010). Comparison Between Modes of Communication for the Deaf and Dumb via e-Learning Through Case Study (sign language and finger spelling in review).. JDCTA. 4. 36-39. 10.4156/jdcta.vol4.issue2.4.
Bhatia, P., Verma, S., and Kaur, S. (2020). Sign Language Generation System Based on Indian Sign Language Grammar. ACM Transactions on Asian Language Information Processing.
Chong, T., and Lee, B. (2018). American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach. https://doi.org/10.3390/s18103554
Hassan, S. T., Abolarinwa, J. A., Alenoghena, C. O., Bala, S. A., David, M., and Farizamin, A. (2017). Intelligent Sign Language Recognition Using Enhanced Fourier Descriptor : A Case of Hausa Sign Language.
Jebali, M., Dakhli, A., and Jemni, M. (2021). Vision based continuous sign language recognition using multimodal sensor fusion. Evolving Systems, 2016. https://doi.org/10.1007/s12530-020-09365-y
Rakibul,H., Al Mahmud,A., Mokhlesur, Yasin, Hossain, and Razib,H . (2021). A Review on the Convolutional Neural Networks (CNN) and Hand- Written Digit Recognition Using Deep Convolutional Neural Network. 6. 1-18.
Rosero-montalvo, P. D., Godoy-trujillo, P., Flores-bosmediano, E., Carrascal-garc, J., Otero-potosi, S., Benitez-pereira, H., & Peluffo-ord, D. H. (2018). Sign Language Recognition Based on Intelligent Glove Using Machine Learning Techniques. 5–9.
Treat, S. (2016). Deaf education: Gallaudet University: How does education and special education is being advanced in Nigeria.
Viswanatha,V., Chandana, R K, and Ramachandra, A.. (2022). Real Time Object Detection System with YOLO and CNN Models: A Review. Xi'an Jianzhu Keji Daxue Xuebao/Journal of Xi'an University of Architecture & Technology. XIV. 10.37896/JXAT14.07/315415.
Zeshan, U. (2004). Sign Languages of the World. Elsevier Ltd., 1989, 358–365.
Copyright (c) 2023 The Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The authors hereby represent and warrant that the paper is original and that they are the authors of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. If in future any violation of any copyright come in notice, then the author will be responsible and not FUOYEJET.
The authors declare that:
- This paper has not been published in the same form elsewhere.
- It will not be submitted anywhere else for publication prior to acceptance/rejection by this Journal.
- A copyright permission is obtained for materials published elsewhere and which require this permission for reproduction.
Furthermore, the copyright after publication belongs to the Author(s) (for articles published in 2020 and beyond) and licensed under the creative commons license CC-BY-NC (http://creativecommons.org/licenses/by-nc/4.0). The copyright covers the right to reproduce and distribute the article, including reprints, translations, photographic reproductions, microform, electronic form (offline, online) or any other reproductions of similar nature.
