Research Information System
Workshops of the International Conference on Computational Science and Its Applications, ICCSA 2025, İstanbul, Turkey, 30 June - 03 July 2025, vol.15886 LNCS, pp.193-211, (Full Text)
This research addresses the gap in direct facial expression-based detection of complex emotional states like stress and fatigue. We propose a novel methodology employing a weighted summation of basic emotion probabilities, outputted by deep learning models, to calculate continuous stress and fatigue scores. Crucially, these emotion weights are empirically justified and grounded in established psychological and neuroscientific literature. Evaluating CNN, hybrid (DDAMFN), and Transformer-based (ViT, BEiT) architectures, our results demonstrate the superior performance of Transformer models, particularly ViT, in aligning with human-annotated ground truth data for stress and fatigue. ViT achieved “almost perfect” Cohen’s Kappa (κ = 0.81) for stress and “substantial” (κ = 0.72) for fatigue, validating the human-relevance of our emotion-based formulation. This study highlights the effectiveness of Transformer architectures and literature-informed emotion weights for direct and accurate stress and fatigue detection from facial expressions, paving the way for real-world applications in monitoring and well-being.