Turkish sign language recognition using kinect skeleton and convolutional neural network Kinect Iskelet ve Evrişimsel Sinir Aǧlari ile Türkçe Işaret Dili Tanima

Unutmaz B., KARACA A. C., Gullu M. K.

27th Signal Processing and Communications Applications Conference, SIU 2019, Sivas, Türkiye, 24 - 26 Nisan 2019 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası:
  • Doi Numarası: 10.1109/siu.2019.8806380
  • Basıldığı Şehir: Sivas
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: Convolutional neural networks, Kinect, Sign language recognition
  • Yıldız Teknik Üniversitesi Adresli: Hayır

Özet

© 2019 IEEE.In this paper, a system that converts Turkish signs to words using convolutional neural networks is presented. Skeleton data obtained by Microsoft's Kinect device is used in the proposed system. User located in front of Kinect sensor makes the signs of corresponding word for a limited time. Afterwards, skeleton joints are extracted. Finally, skeleton points on consecutive frames are merged, and Word classification is performed by convulational neural network. In experimental studies, evaluations are carried out with our own dataset and performance of the proposed method is compared with various classification methods. Moreover, the effect of being closer or further to the camera and movements in different speeds are also investigated. In the lights of the experimental results, it is seen that proposed convolutional neural network gives better performances than other classifiers such as Gaussian SVM, Linear SVM, weighted KNN and decision trees.