Enhancing harmful content detection in memes using multimodal machine learning models

• Ano de defesa: 2025
• Autor(a) principal: Queiroz Hermida, Paulo Cezar de
• Outros Autores: http://lattes.cnpq.br/3158714298481379, https://orcid.org/0000-0001-6875-8465
• Tipo de documento: Tese
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Universidade Federal do Amazonas; Instituto de Computação; Brasil; UFAM; Programa de Pós-graduação em Informática
• Palavras-chave em Português: CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO; Harmful memes; Multimodal models; Harmful content detection
• Link de acesso: https://tede.ufam.edu.br/handle/tede/10697

This thesis focuses on detecting harmful content in memes using advanced machine learning methods. It begins with a literature review, identifying the strengths, weaknesses, and challenges of current approaches while introducing a new taxonomy to facilitate method comparison. The research presents an improvement to canonical multimodal transformer models by integrating Compact Parameter Blocks into the encoder segments, achieving superior performance compared to more complex techniques. Additionally, it explores the use of generative models, such as Multimodal Large Language Models (MLLMs), to detect aggressive memes through specific prompts. The results indicate that while these models can identify harmful content, their performance declines when high-level multimodal reasoning is required. This research contributes to the field by enhancing detection methods and exploring new generative model-based approaches, aiming to create safer online environments while preserving freedom of expression.