Multimodal quantification of mental illness through machine learning

• Autor(a) principal: Pires, Márcia Jesus
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Texto Completo: http://hdl.handle.net/10773/38711
• topic: Deep learning; Depression prediction; Machine learning; Mental health; Natural language processing

Approximately 280 million people worldwide suffer from depression - the foremost cause of mental-health illness - and the tendency is for this number to continue to grow. This depressive disorder affects an individual psychologically and physically, leading, in the worst case, to the loss of lives. Therefore, early detection of depression is critical for rapid assessment and intervention, which can contribute to minimizing the escalation of the disorder. However, the current diagnosis methods are limited and subjective, depending almost entirely on verbal reports. Thus, there is an urgent need to develop systematic strategies to monitor and diagnose depression. In that scope, this dissertation focus on developing machine learning and deep learning models capable of automatically detecting the presence and quantifying the severity of depression using verbal and non-verbal indicators. From semi-structured clinical interviews, it was possible to analyze audio and video recordings and transcripts from participants’ speech with the intuition of extracting the most revealing characteristics of depression inside a modality. Those extracted features allowed the development of supervised machine learning unimodal models and deep learning ones that later culminated in a multimodal model. Finally, the results confirmed the efficiency of fusing different modalities, and the multimodal model predicted depression presence with an F1-Score of 0.83. At the same time, the severity measure of depression obtained an RMSE of 5.91.