A Real-Time System For Recognition Of American Sign Language By Using Deep Learning

TAŞKIRAN M., KILLIOĞLU M., KAHRAMAN N.

41st International Conference on Telecommunications and Signal Processing (TSP), Athens, Yunanistan, 4 - 06 Temmuz 2018, ss.258-261 identifier identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/tsp.2018.8441304
  • Basıldığı Şehir: Athens
  • Basıldığı Ülke: Yunanistan
  • Sayfa Sayıları: ss.258-261
  • Anahtar Kelimeler: american sign language, classification, convex hull, convolutional neural network, deep learning, real-time
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

Deaf people use sign languages to communicate with other people in the community. Although the sign language is known to hearing-impaired people due to its widespread use among them, it is not known much by other people. In this article, we have developed a real-time sign language recognition system for people who do not know sign language to communicate easily with hearing-impaired people. The sign language used in this paper is American sign language. In this study, the convolutional neural network was trained by using dataset collected in 2011 by Massey University, Institute of Information and Mathematical Sciences, and 100% test accuracy was obtained. After network training is completed, the network model and network weights are recorded for the real-time system. In the real-time system, the skin color is determined for a certain frame for hand use, and the hand gesture is determined using the convex hull algorithm, and the hand gesture is defined in real-time using the registered neural network model and network weights. The accuracy of the real-time system is 98.05%.