Micro-expression recognition using facial landmarks

• Main Author: Lopes, Fernando Júnior Loureiro
• Publication Date: 2023
• Format: Master thesis
• Language: eng
• Source: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Download full: http://hdl.handle.net/10773/41819
• topic: Emotions; Micro-expressions; Action units; Facial landmarks; Emotion recognition

Facial expressions have long been recognized as a crucial element of non-verbal communication, making them a fundamental aspect to express human emotions. Given their effectiveness at conveying emotions, some individuals may attempt to conceal or mask their genuine feelings by controlling their facial expressions. Nonetheless, it is important to note that genuine emotions cannot be concealed seamlessly, they consistently manifest through subtle muscle movements. This emphasizes the importance of spotting these facial cues to unveil authentic emotions. This dissertation aims to explore subtle facial muscle movements using facial landmarks to describe micro-expressions These micro-expressions are brief combinations of individual muscle movements. To accomplish this, we conducted a comprehensive exploration and comparison of state-of-the-art solutions for landmark detection to identify the most suitable option for micro-expression detection. With an effective landmark detector in place and careful selection of facial landmarks, we extracted features from facial expressions using geometric methods. These extracted features allowed us to discern patterns associated with specific human emotions such as happiness, sadness, anger, surprise, fear, and disgust. As a result, we developed a fully functional proofof-concept processing pipeline for micro-expression spotting. This pipeline integrates facial action unit (FAU) extraction from video and emotions classification module. The classification module explored standard machine learning models (e.g. SVM and random forest) to identify emotions from FAU. The results indicate that future work is needed to enhance emotion classification, especially for happiness, repression, and disgust expressions.