Open Access

Ömer Sümer

• In social signal processing and computer vision, there has been increasing number of studies which are related with social and behavioural sciences to some extent in last years. Affective state of human has very significant potential in many application areas such as evaluating market trends, understanding the decision-making, interpreting social interactions and their underlying background, and so on. Among the agents that make our emotions understandable, the facial expressions are the most prominent and descriptive sign of a humans's affective state. This thesis presents a literature survey on the state-of-the-art of facial expression recognition, comparison of different approaches in automatic analysis of emotions, and proposes a new embedded framework for facial expression recognition problem. Although there have been large number of studies in facial expression recognition, the number of ``affective'' embedded systems are fairly scarce. In this study, an efficient embeddedIn social signal processing and computer vision, there has been increasing number of studies which are related with social and behavioural sciences to some extent in last years. Affective state of human has very significant potential in many application areas such as evaluating market trends, understanding the decision-making, interpreting social interactions and their underlying background, and so on. Among the agents that make our emotions understandable, the facial expressions are the most prominent and descriptive sign of a humans's affective state. This thesis presents a literature survey on the state-of-the-art of facial expression recognition, comparison of different approaches in automatic analysis of emotions, and proposes a new embedded framework for facial expression recognition problem. Although there have been large number of studies in facial expression recognition, the number of ``affective'' embedded systems are fairly scarce. In this study, an efficient embedded framework is implemented on a system-on-chip (SoC) development board. Many application areas of facial expression recognition systems necessitate the mobility, and embedded platforms which have both hardware and software development tools, as well as low power consumption and increased adaptivity. In this study, different feature extraction methods such as local binary pattern (LBP), local ternary pattern (LTP) and Gabor filters are compared using different extraction strategies and varied kernel functions and parameters in learning phase, support vector machines (SVM). In embedded framework of facial expression system, local binary patterns and support vector machines-based methodology is preferred, because of its higher accuracy and time performance. Besides OpenCV implementation on embedded linux operating system, Zynq-7000 all programmable SoC is used to measure the performance of LBP feature extraction. Our final system has capable of facial expression recognition in both static images and video sequences at 4-5 fps.…