Turkish sign language recognition using kinect skeleton and convolutional neural network Kinect Iskelet ve Evrişimsel Sinir Aǧlari ile Türkçe Işaret Dili Tanima


Unutmaz B., KARACA A. C. , Gullu M. K.

27th Signal Processing and Communications Applications Conference, SIU 2019, Sivas, Turkey, 24 - 26 April 2019 identifier

  • Publication Type: Conference Paper / Full Text
  • Volume:
  • Doi Number: 10.1109/siu.2019.8806380
  • City: Sivas
  • Country: Turkey
  • Keywords: Convolutional neural networks, Kinect, Sign language recognition

Abstract

© 2019 IEEE.In this paper, a system that converts Turkish signs to words using convolutional neural networks is presented. Skeleton data obtained by Microsoft's Kinect device is used in the proposed system. User located in front of Kinect sensor makes the signs of corresponding word for a limited time. Afterwards, skeleton joints are extracted. Finally, skeleton points on consecutive frames are merged, and Word classification is performed by convulational neural network. In experimental studies, evaluations are carried out with our own dataset and performance of the proposed method is compared with various classification methods. Moreover, the effect of being closer or further to the camera and movements in different speeds are also investigated. In the lights of the experimental results, it is seen that proposed convolutional neural network gives better performances than other classifiers such as Gaussian SVM, Linear SVM, weighted KNN and decision trees.