Research Information System
Thesis Type: Doctorate
Institution Of The Thesis: Yildiz Technical University, Graduate School Of Natural And Applied Sciences, Turkey
Approval Date: 2022
Thesis Language: Turkish
Student: ONUR CAN KURBAN
Supervisor: Tülay Yıldırım
Abstract:
In recent years, the analysis of human behaviour has become one of the most popular
research areas. This analysis aims to understand people’s behaviour over a period of
time using measurable behavioural and physical movement information. Behaviour
analysis is classified as gestures, gestures, interactions, activity or person recognition
and verification, depending on the time of observation. This analysis explores not
only a person’s behaviour but also interpersonal interactions and interactions between
persons-objects. The behavioural movements studied can be further subdivided into
subcategories such as upper or lower body, hand or arm movements, and facial
expressions. Recognition of complex human activities makes it possible to realize
some important applications. Automatic surveillance systems in public places such
as airports and subway stations to detect abnormal activity; real-time monitoring
of patients, the elderly and children; gesture-based human-computer interfaces; and
imaging-based smart environments are examples of these applications.
In the first stage of this study, research was conducted on a new energy image method,
which is created according to the changes occurring during the movement, which can
reveal the changes in the movement more. In addition, the effects of behavioural
biometrics on multi-biometric performance were investigated. As a result of these
studies, a new energy image generation approach has been proposed for action
recognition and person identification based on the motion sequence information in
masked depth video streams obtained from RGB-D data. In the proposed method,
the first frame is referenced to determine the changes that occur during a motion sequence and the correlation coefficients between all frames are calculated. The
obtained coefficients are used to create a temporal pattern compatible with the motion.
The energy images generated by this function, called the adaptive temporal template,
will be able to better represent movements compared to existing temporal patterns
in the literature that emphasize specific periods. In this way, a practical method is
proposed that provides a better representation of actions in energy images and reduces
the cost of processing as it eliminates the need for fusion of multiple TTs. In addition,
RGB-D-based studies are mostly used for motion recognition in the literature. Another
contribution of this study is that the existing and proposed methods are examined in
the detection of persons from the movement.
Energy images created with the function proposed in the thesis emphasize the ranges
of motion with more variation while suppressing the intervals with less variation. In
order to understand the distinguishing features, the energy images obtained using the
proposed function are given as input to convolutional neural networks and different
hand-crafted classifiers. The proposed method was observed on BodyLogin, NATOPS
and SBU Kinect datasets and compared with existing methods. Apart from this, a new
dataset consisting of behavioural, thermal and physical data was created to examine
the effects of behavioural and energy image data on multi-biometrics and the proposed
method was tested on this new dataset.
The results show that the proposed adaptive temporal template method provides
higher performance and shorter processing time compared to the templates available
in the literature. Person identification results showed that images obtained with
RGB-D sensors can be used effectively for person identification.