Facial Expression Recognition Via Multi Semantic Diffusion Model on Imbalanced Datasets

This paper presents a novel facial expression recognition approach based on multiple semantic taxonomies learning on the imbalanced datasets. Recent studies on imbalanced data always concern how to homogenize the data volume between different categories, presenting strategies like minority over-sampling and majority balance cascading, etc. In this paper, we try to pay more attention in high-level semantic characterization of facial expression, using more discriminative and conceptual attributes to describe samples in the case of unbalanced sets. To fully exploit the semantic information contained in the small volume samples, we develop an Analytic Hierarchical Model (AHM) method based on facial Action Unit (AU), to enforce a discriminative mapping from the image feature space to a multi-semantic space with taxonomic relations. We apply convolutional neural networks to capture the low-level image feature, and then use dictionary learning algorithm for reconstruction of images in semantic space, in order to prevent deviation from individual identity. Experiments performed on RAF-DB, FER2013 and SFEW expression databases show that the proposed method is robust to facial expression recognition in the wild.