Aplicação de técnicas de inteligência computacional para análise da expressão facial em reconhecimento de sinais de libras

• Ano de defesa: 2016
• Autor(a) principal: Tamires Martins Rezende
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: por
• Instituição de defesa: Universidade Federal de Minas Gerais; UFMG
• Palavras-chave em Português: Reconhecimento de sinais; Kinect; k-NN; Sensor RGB-D; Libras; SVM; LBP; Engenharia elétrica; Expressão facial; Detecção de sinais; Língua brasileira de sinais
• Link de acesso: http://hdl.handle.net/1843/RAOA-BC5HK9

The automatic recognition of facial expressions is a complex problem that requires the application of Computational Intelligence techniques, especially those related to Pattern Recognition. The use of these techniques aims to establish an approach that allows identify signs of the Brazilian Sign Language, known as Libras, through one of its phonological parameters: non-manual expressions. These expressions are formed by movement of the face, eyes, head and/or trunk. The main objective of the present research was to measure the importance of facial expression during the execution of sign in Libras and to verify if only the change in physiognomy is enough to identify one. From this premise, a methodology for the automatic recognition of Libras signs was structured and validated by a database composed of 10 Libras signs recorded by a RGB-D (Kinect) sensor. This sign database was built for this application and in it each sign selected for its composition was executed by only one flag. The Libras sign database provides the coordinates (x,y) of the 121-point face position and the videos of each recording of each signal. From this available information, the following steps were implemented: (i) face detection and clipping, which is the region of interest in this work; (ii) summarization videos with face images using the concept of maximizing diversity in terms of temporal distance and color difference in RGB pattern between frames. This step was necessary to eliminate redundant information and through it the five most significant frames of the recordings of each signal were obtained; (iii) creation of two characteristic vectors: one from the concatenation of the 121 cartesian points available in the sign database and another from the information obtained by applying the LBP (Binary Local Patterns) texture descriptor in each of the significant frames; and (iv) classification of the signs by applying k-NN (k-nearest neighbors) and SVM (Support Vector Machine). The best parameters for these classifiers (respectively the parameter k of the first, and C and of the second) were obtained from cross validation. The classification of the signs of the database created by means of the characteristic generated by the application of the descriptor LBP in the most significant pictures of the videos of the recordings of each sign had better performance than the characteristic derived from the concatenation of cartesian points. In relation to the classifiers, the SVM returned better hit rates. Thus, the mean accuracy of sign recognition obtained from the analysis of the methodology proposed here was of 95.3% evidencing the potentiality of the proposed model. This work contributes to the growth of studies that involve the visual aspects of the structure of Libras and focuses on the importance of facial expression in the identification of signs in an automated way.