Application of Convolutional Neural Network (CNN) for Web-Based Translation of Indonesian Text into Sign Language
DOI:
https://doi.org/10.33005/jasid.v1i2.12Keywords:
Sign language, Convolutional Neural Network (CNN), ASL Alphabet, Streamlit, Machine LearningAbstract
Communication for the deaf and hard of hearing is often hindered by the limited number of sign language interpreters. This research aims to develop a web-based text-to-text sign language translation system using Convolutional Neural Networks (CNN) to bridge this communication gap. The system is built with the ASL Alphabet dataset containing 87,000 images from 29 classes (A-Z, SPACE, DELETE, NOTHING). The CNN model was designed with three convolutional layers and trained for 15 epochs using 80% of the data, while 20% of the data was used for testing. The user interface was developed using Streamlit for ease of use. Training results showed a training accuracy of 98.96% and a validation accuracy of 98.61% at the 15th epoch. Model evaluation yielded an overall accuracy of 98%, with high precision, recall, and F1-score values for most classes. This research demonstrates the significant potential of CNN in developing automatic sign language translators, which is expected to improve information accessibility and inclusivity for the deaf community.
References
R. Brindha, P. Renukadevi, D. Vathana, and J. D, “Sign Language Interpreter,” in 2022 International Conference on Inventive Computation Technologies (ICICT), IEEE, Jul. 2022, pp. 1024–1028. doi: 10.1109/ICICT54344.2022.9850486.
National Geographic Society, “Sign Language,” National Geographic Society. Accessed: Jun. 05, 2025. [Online]. Available: https://education.nationalgeographic.org/resource/sign-language/?utm_source=chatgpt.com
World Federation of the Deaf (WFD), “The Legal Recognition of National Sign Languages,” World Federation of the Deaf. Accessed: Jun. 05, 2025. [Online]. Available: https://wfdeaf.org/the-legal-recognition-of-national-sign-languages/?utm_source=chatgpt.com&_s=indonesia
R. Maheswara, “Jumlah Penyandang Disabilitas di Indonesia 2022 Berdasarkan Jenis Keterbatasan,” Humaniora. Accessed: Jun. 05, 2025. [Online]. Available: https://dataloka.id/humaniora/1967/jumlah-penyandang-disabilitas-di-indonesia-2022-berdasarkan-jenis-keterbatasan/?utm_source=chatgpt.com#google_vignette
D. Ponnappa and B. G. Jairam, “A Comparative Study on Sign Language Translation Using Artificial Intelligence Techniques,” 2023, pp. 359–368. doi: 10.1007/978-981-19-8742-7_30.
S. Meshram, R. Singh, P. Pal, and S. K. Singh, “Convolution Neural Network based Hand Gesture Recognition System,” in 2023 Third International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), IEEE, Jan. 2023, pp. 1–5. doi: 10.1109/ICAECT57570.2023.10118267.
O. Dwi Nurhayati, D. Eridani, and M. Hafiz Tsalavin, “SISTEM ISYARAT BAHASA INDONESIA (SIBI) METODE CONVOLUTIONAL NEURAL NETWORK SEQUENTIAL SECARA REAL TIME A REAL-TIME INDONESIAN LANGUAGE SIGN SYSTEM USING THE CONVOLUTION NEURAL NETWORK METHOD,” Jurnal teknologi Informasi dan Ilmu Komputer (JTIIK)), vol. 9, pp. 819–828, 2022, doi: 10.25126/jtiik.202294787.
R. Haris Alfikri et al., “PEMBANGUNAN APLIKASI PENERJEMAH BAHASA ISYARAT DENGAN METODE CNN BERBASIS ANDROID,” 2022. [Online]. Available: https://ejurnal.teknokrat.ac.id/index.php/teknoinfo/index
I. Muslim Pramono, imatun Niswati, and A. Agustina, “MODEL PENERJEMAH BAHASA ISYARAT DENGAN METODE CONVOLUTIONAL NEURAL NETWORK (CNN),” jakarta, Jan. 2024.
F. Chollet and Others, ‘Keras’, 2015. [Online]. Available: https://keras.io.
D. P. Kingma and J. Ba, ‘Adam: A Method for Stochastic Optimization’, arXiv [cs.LG]. 2017.
A. Muhaimin, D. D. Prastyo and H. Horng-Shing Lu, "Forecasting with Recurrent Neural Network in Intermittent Demand Data," 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, 2021, pp. 802-809, doi: 10.1109/Confluence51648.2021.9376880.
