Two-Stage Recognition and beyond for Compound Facial Emotion Recognition

Creative Commons License

Kaminska D., Aktas K., Rizhinashvili D., Kuklyanov D., Sham A. H., Escalera S., ...More

ELECTRONICS, vol.10, no.22, 2021 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 10 Issue: 22
  • Publication Date: 2021
  • Doi Number: 10.3390/electronics10222847
  • Journal Name: ELECTRONICS
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Keywords: compound emotion recognition, facial expression recognition, dominant and complementary emotion recognition, deep learning, EXPRESSIONS, MODEL
  • Yıldız Technical University Affiliated: No


Facial emotion recognition is an inherently complex problem due to individual diversity in facial features and racial and cultural differences. Moreover, facial expressions typically reflect the mixture of people's emotional statuses, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes 31,250 facial images with different emotions of 115 subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner's approach-a two-stage recognition method (1st stage, coarse recognition; 2nd stage, fine recognition), which enhances the classification of symmetrical emotion labels.