Deep Learning for Micro-Expression Recognition

Micro-expressions (MEs) are involuntary facial movements revealing people’s hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently with the success of Deep Learning (DL) in various tasks neural networks have received increasing interest in MER. Different from macro-expressions MEs are spontaneous subtle and rapid facial movements leading to difficult data collection and annotation thus publicly available datasets are usually small-scale. Currently various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL including datasets each step of the deep MER pipeline and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally ethical considerations in MER are discussed. To the best of our knowledge this is the first survey of deep MER methods and this survey can serve as a reference point for future MER research.