Landmark-Guided Attention for Interpretable Facial Emotion Recognition Using Modified ResNet-18

Abstract

A facial emotion recognition approach that integrates explicit spatial attention based on facial landmark information into a deep learning pipeline. By employing MediaPipe FaceMesh to detect facial landmarks, the system generates attention heatmaps that highlight emotionally relevant regions such as the eyes, eyebrows, and mouth corners. These attention maps are fused with preprocessed grayscale face images to form a four-channel input, which is then classified using a modified ResNet-18 architecture. Experiments conducted on a diverse in-house dataset encompassing six emotion categories demonstrate that the proposed model achieves competitive accuracy while providing clear interpretability through visualized attention. Comparative analyses reveal that Apart from making the attention of the neural network clearer, landmark-guided attention enhances the network performance on subtle or difficult expressions. The approach provides strong and understandable results, highlighting its value for emotion AI in the real world. Future work can extend this framework to incorporate multimodal signals, more dynamic attention mechanisms, and more data augmentation for even wider deployments.