Human and action recognition using adaptive energy images

KURBAN O. C., ÇALIK N., YILDIRIM T.

PATTERN RECOGNITION, vol.127, 2022 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 127
  • Publication Date: 2022
  • Doi Number: 10.1016/j.patcog.2022.108621
  • Journal Name: PATTERN RECOGNITION
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Applied Science & Technology Source, BIOSIS, Computer & Applied Sciences, INSPEC, MLA - Modern Language Association Database, zbMATH
  • Keywords: Motion recognition, Human recognition, Correlation coefficients, Deep learning, Behavioral biometrics, RGB-D
  • Yıldız Technical University Affiliated: Yes

Abstract

In this paper, we propose a new temporal template approach for action recognition and person identification based on motion sequence information in masked depth video streams obtained from RGB-D data. This new representation creates a membership function that models the change in motion based on the correlation between frames that occur during motion flow. The energy images created with this function emphasize the intervals of motion with more change, while the intervals with less change are suppressed. To understand the distinctive features, the obtained energy images by using the proposed function are given as input to the convolutional neural networks and different handcrafted classifiers. The proposed method was observed on the BodyLogin, NATOPS, and SBU Kinect datasets and compared with the existing temporal templates and recent methods. The results indicate that the proposed method provides both higher performance and better motion representation. (c) 2022 Elsevier Ltd. All rights reserved.